Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reporter is not starting since 1.5.1 for worker processes #184

Closed
kwent opened this issue Aug 16, 2023 · 11 comments
Closed

Reporter is not starting since 1.5.1 for worker processes #184

kwent opened this issue Aug 16, 2023 · 11 comments

Comments

@kwent
Copy link

kwent commented Aug 16, 2023

Hi,

This PR is preventing the reporter to start in all our worker processes (sidekiq specifically).

Reversing to 1.5.0 fixes the issue on our side.

Regards

@kwent kwent changed the title https://github.com/judoscale/judoscale-ruby/pull/180 Report is not starting since 1.5.1 for worker processes Aug 16, 2023
@kwent kwent changed the title Report is not starting since 1.5.1 for worker processes Reporter is not starting since 1.5.1 for worker processes Aug 16, 2023
@jon-sully
Copy link
Member

Hey @kwent, thanks for the report! Could you share with us what command you use for starting those Sidekiq processes?

@kwent
Copy link
Author

kwent commented Aug 16, 2023

Procfile

release: bin/migrate
web: bundle exec puma
worker: IS_SCHEDULER=true bundle exec sidekiq -t 20 -C config/sidekiq.yml

@adamlogic
Copy link
Collaborator

@kwent That PR suppresses the reporter when running in a Rake task or Rails console, and that shouldn't be the case when running bundle exec sidekiq.

Would you mind installing 1.5.1 (or 1.5.2, shouldn't matter) locally so we could see the output of the following statements? Paste them into an initializer (any initializer) and start Sidekiq via bundle exec sidekiq or by running the worker process from your Procfile.

puts "=" * 80
puts "console: #{defined?(::Rails::Console)}"
puts "rake: #{defined?(::Rake)}"
puts "rake app: #{defined?(::Rake) && Rake.respond_to?(:application)}"
puts "tasks: #{defined?(::Rake) && Rake.respond_to?(:application) && Rake.application.top_level_tasks.inspect}"
puts "=" * 80

@kwent
Copy link
Author

kwent commented Aug 17, 2023

Weirdly this is outputting same values.

using 1.5.0

================================================================================
console: constant
rake: constant
rake app: true
tasks: []
================================================================================

using 1.5.2

================================================================================
console: constant
rake: constant
rake app: true
tasks: []
================================================================================

@adamlogic
Copy link
Collaborator

Weirdly this is outputting same values

Oh right, that makes sense. v1.5.1 is the first version we started looking at those values to suppress the reporter, but the values would be the same regardless of our gem.

Thanks for doing this! It's really helpful, and I should have a fix out later today.

@adamlogic
Copy link
Collaborator

Nope, I take that back. I thought we were treating the empty array of rake tasks as truthy, but we're already calling .any? on it.

Can you run one more thing for me? Doesn't matter what version of the gem.

puts "=" * 80
puts "generate: #{defined?(::Rails::Command::GenerateCommand)}"
puts "=" * 80

@kwent
Copy link
Author

kwent commented Aug 17, 2023

Sure thing

================================================================================
generate:
================================================================================

@adamlogic
Copy link
Collaborator

Thanks, that's what I would expect, so I'm still baffled why the reporter would have been suppressed.

Do you have access to your application logs from when 1.5.1 was preventing the reporter from starting? I'd love to see the Judoscale logs (prefixed with [Judoscale]) from when your worker dyno booted.

@kwent
Copy link
Author

kwent commented Aug 17, 2023

Unfortunately way too hard for me to find this information ( we have lot of logs ).

@adamlogic
Copy link
Collaborator

Yep, I understand.

I've confirmed that we have several customers using v1.5.2 with Sidekiq processes reporting as expected, so the issue appears to be specific to your app.

If you decide to try the upgrade again and are able to capture those logs, please report back.

@kwent
Copy link
Author

kwent commented Jul 30, 2024

We were indeed using safer_rails_console Cf. #192. We upgraded to latest judoscale version and don't see this issue anymore. 🙏🏻

@kwent kwent closed this as completed Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants