Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

provide better logging (on errors and sql_last_value tracking) #54

Open
kares opened this issue Nov 27, 2020 · 1 comment
Open

provide better logging (on errors and sql_last_value tracking) #54

kares opened this issue Nov 27, 2020 · 1 comment
Assignees

Comments

@kares
Copy link
Contributor

kares commented Nov 27, 2020

esp. in scheduler mode we get some logging but the format is different from LS' looks messy and goes to stderr :

[2020-11-27T17:10:11,355][INFO ][logstash.inputs.jdbc     ] (0.002910s) SELECT * FROM employee WHERE updated_at IS NOT NULL AND updated_at > '1970-01-01 01:00:00.000000+0100' ORDER BY updated_at
{ 2054 rufus-scheduler intercepted an error:
  2054   job:
  2054     Rufus::Scheduler::CronJob "*/1 * * * * *" {}
  2054   error:
  2054     2054
  2054     RuntimeError
  2054     42
  2054       /work/elastic/plugins/logstash-integration-jdbc/lib/logstash/plugin_mixins/jdbc/value_tracking.rb:123:in `write'
  2054       /work/elastic/plugins/logstash-integration-jdbc/lib/logstash/plugin_mixins/jdbc/value_tracking.rb:46:in `write'
  2054       /work/elastic/plugins/logstash-integration-jdbc/lib/logstash/inputs/jdbc.rb:331:in `execute_query'
  2054       /work/elastic/plugins/logstash-integration-jdbc/lib/logstash/inputs/jdbc.rb:285:in `block in run'
  2054       /opt/local/rvm/gems/jruby-9.2.12.0@ls/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:234:in `do_call'
  2054       /opt/local/rvm/gems/jruby-9.2.12.0@ls/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:258:in `do_trigger'
  2054       /opt/local/rvm/gems/jruby-9.2.12.0@ls/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:300:in `block in start_work_thread'
  2054       /opt/local/rvm/gems/jruby-9.2.12.0@ls/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:299:in `block in start_work_thread'
  2054       org/jruby/RubyKernel.java:1442:in `loop'
  2054       /opt/local/rvm/gems/jruby-9.2.12.0@ls/gems/rufus-scheduler-3.0.9/lib/rufus/scheduler/jobs.rb:289:in `block in start_work_thread'
  2054   tz:
  2054     ENV['TZ']: 
  2054     Time.now: 2020-11-27 17:10:11 +0100
  2054   scheduler:
  2054     object_id: 2050
  2054     opts:
  2054       {:max_work_threads=>1}
  2054       frequency: 0.3
  2054       scheduler_lock: #<Rufus::Scheduler::NullLock:0x31b02296>
  2054       trigger_lock: #<Rufus::Scheduler::NullLock:0x1a690b63>
  2054     uptime: 1.317325 (1s317)
  2054     down?: false
  2054     threads: 2
  2054       thread: #<Thread:0x3328fce3>
  2054       thread_key: rufus_scheduler_2050
  2054       work_threads: 1
  2054         active: 1
  2054         vacant: 0
  2054         max_work_threads: 1
  2054       mutexes: {}
  2054     jobs: 1
  2054       at_jobs: 0
  2054       in_jobs: 0
  2054       every_jobs: 0
  2054       interval_jobs: 0
  2054       cron_jobs: 1
  2054     running_jobs: 1
  2054     work_queue: 0
} 2054 .

we should also provide logging around last_value updates, optionally traces around rows read.

@kares kares changed the title provide better logging (on sql_last_value and errors) provide better logging (on errors and sql_last_value tracking) Nov 27, 2020
@kares kares self-assigned this May 2, 2022
@kares
Copy link
Contributor Author

kares commented May 2, 2022

the error logging (redirection) part has been resolved by: #102

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants