Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graylog sidecar validate runs only 30 seconds #431

Closed
mpiscaer opened this issue Jun 27, 2022 · 2 comments · Fixed by #435
Closed

Graylog sidecar validate runs only 30 seconds #431

mpiscaer opened this issue Jun 27, 2022 · 2 comments · Fixed by #435

Comments

@mpiscaer
Copy link

I'm have a sidecar running that manages a logstash config. But the validation often takes more that 30 seconds.

At the code https://github.com/Graylog2/collector-sidecar/blob/master/backends/backend.go#L171 I don't see that you can configure this.

Am I correct on this, if so it would be great to able to configure the timeout of the validation.

@boosty boosty changed the title Greylog sidecar validate runs only 30 seconds Graylog sidecar validate runs only 30 seconds Jul 4, 2022
@mpfz0r mpfz0r self-assigned this Aug 1, 2022
@mpfz0r
Copy link
Contributor

mpfz0r commented Aug 1, 2022

@mpiscaer @reg1oDATA I'm skeptical. Why would the configuration test take more than 30 seconds?
Could you verify this by running the validation command manually?
e.g. /path/to/your/winlogbeat test config -c /path/to/config.yml ?

I don't mind adding a config option to extend the default timeout, if that helps to solve the actual problem.

@mpiscaer
Copy link
Author

mpiscaer commented Aug 1, 2022

When I run the the logstash test by hand I get the following:

root@sidecar-logstash:~# time /usr/share/logstash/bin/logstash --config.test_and_exit --path.settings /etc/logstash/
Using bundled JDK: /usr/share/logstash/jdk
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Sending Logstash logs to /var/log/logstash which is now configured via log4j2.properties
[2022-08-01T09:53:47,078][INFO ][logstash.runner          ] Log4j configuration path used is: /etc/logstash/log4j2.properties
[2022-08-01T09:53:47,089][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.17.1", "jruby.version"=>"jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 11.0.13+8 on 11.0.13+8 +indy +jit [linux-x86_64]"}
[2022-08-01T09:53:47,092][INFO ][logstash.runner          ] JVM bootstrap flags: [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djruby.compile.invokedynamic=true, -Djruby.jit.threshold=0, -Djruby.regexp.interruptible=true, -XX:+HeapDumpOnOutOfMemoryError, -Djava.security.egd=file:/dev/urandom, -Dlog4j2.isThreadContextMapInheritable=true]
[2022-08-01T09:53:49,868][INFO ][org.reflections.Reflections] Reflections took 75 ms to scan 1 urls, producing 119 keys and 417 values 
Configuration OK
[2022-08-01T09:53:51,745][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

real    0m31.622s                                                   
user    1m50.823s                                                                                                                       
sys     0m4.383s                                                    
root@sidecar-logstash:~# 

In in config are a couple of 7 inputs 1 small filter and 1 output.

It would be great if you can add a setting for this.

mpfz0r added a commit that referenced this issue Aug 1, 2022
Intoduce a new config option: `collector_validation_timeout = 1m`
Also increase the default timeout from 30s to 1m
since 30s seems to low for some use cases.

Fixes #431
mpfz0r added a commit that referenced this issue Aug 2, 2022
Intoduce a new config option: `collector_validation_timeout = 1m`
Also increase the default timeout from 30s to 1m
since 30s seems to low for some use cases.

Fixes #431
mpfz0r added a commit that referenced this issue Aug 2, 2022
Intoduce a new config option: `collector_validation_timeout = 1m`
Also increase the default timeout from 30s to 1m
since 30s seems to low for some use cases.

Fixes #431

(cherry picked from commit 56bd153)
mpfz0r added a commit that referenced this issue Aug 2, 2022
Intoduce a new config option: `collector_validation_timeout = 1m`
Also increase the default timeout from 30s to 1m
since 30s seems to low for some use cases.

Fixes #431

(cherry picked from commit 56bd153)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants