Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issues when running on macOs #178

Closed
ArglosStudios opened this issue Mar 15, 2024 · 17 comments
Closed

issues when running on macOs #178

ArglosStudios opened this issue Mar 15, 2024 · 17 comments

Comments

@ArglosStudios
Copy link

Hello, I am running a project with solid_queue.

When doing some development on my windows pc with WSL, solid_queue runs greatly! However, when running on my macbook, I get either get file descriptors or weird reading errors:

Bad file descriptors:

10:16:17 jobs.1 | [SolidQueue] Restarting fork[5568] (status: 1)
10:16:17 jobs.1 | [SolidQueue] Starting Worker(pid=5592, hostname=Carloss-MacBook-Pro.local, metadata={:polling_interval=>0.3, :queues=>"*", :thread_pool_size=>4})
10:16:48 jobs.1 | /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/interruptible.rb:21:in `wait_readable': Bad file descriptor (Errno::EBADF)
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/interruptible.rb:21:in `interruptible_sleep'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/worker.rb:29:in `run'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/runnable.rb:48:in `block (2 levels) in do_start_loop'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.3/lib/active_support/execution_wrapper.rb:92:in `wrap'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/app_executor.rb:7:in `wrap_in_app_executor'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/runnable.rb:47:in `block in do_start_loop'
10:16:48 jobs.1 |       from <internal:kernel>:187:in `loop'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/runnable.rb:44:in `do_start_loop'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/runnable.rb:39:in `start_loop'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/runnable.rb:15:in `start'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/supervisor.rb:123:in `block in start_fork'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/supervisor.rb:122:in `fork'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/supervisor.rb:122:in `start_fork'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/supervisor.rb:152:in `replace_fork'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/supervisor.rb:134:in `block in reap_and_replace_terminated_forks'
10:16:48 jobs.1 |       from <internal:kernel>:187:in `loop'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/supervisor.rb:130:in `reap_and_replace_terminated_forks'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/supervisor.rb:51:in `block in supervise'
10:16:48 jobs.1 |       from <internal:kernel>:187:in `loop'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/supervisor.rb:47:in `supervise'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/supervisor.rb:26:in `start'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/supervisor.rb:14:in `start'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/tasks.rb:4:in `block (2 levels) in <main>'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `block in execute'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `each'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `execute'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `synchronize'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `invoke_with_call_chain'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:188:in `invoke'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:182:in `invoke_task'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block (2 levels) in top_level'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `each'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block in top_level'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:147:in `run_with_threads'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:132:in `top_level'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:83:in `block in run'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:208:in `standard_exception_handling'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:80:in `run'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/exe/rake:27:in `<top (required)>'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/bin/rake:25:in `load'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/bin/rake:25:in `<top (required)>'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/cli/exec.rb:58:in `load'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/cli/exec.rb:58:in `kernel_load'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/cli/exec.rb:23:in `run'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/cli.rb:451:in `exec'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/cli.rb:34:in `dispatch'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/cli.rb:28:in `start'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.3/exe/bundle:28:in `block in <top (required)>'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/3.3.0/bundler/friendly_errors.rb:117:in `with_friendly_errors'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/bundler-2.5.3/exe/bundle:20:in `<top (required)>'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/bin/bundle:25:in `load'
10:16:48 jobs.1 |       from /Users/carlos/.rbenv/versions/3.3.0/bin/bundle:25:in `<main>'
10:16:49 jobs.1 | [SolidQueue] Restarting fork[5592] (status: 1)

end of file reached error:

10:25:29 jobs.1 | <internal:io>:63:in `read_nonblock': end of file reached (EOFError)
10:25:29 jobs.1 | 	from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/interruptible.rb:22:in `block in interruptible_sleep'
10:25:29 jobs.1 | 	from <internal:kernel>:187:in `loop'
10:25:29 jobs.1 | 	from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/interruptible.rb:22:in `interruptible_sleep'
10:25:29 jobs.1 | 	from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/worker.rb:29:in `run'
10:25:29 jobs.1 | 	from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/runnable.rb:48:in `block (2 levels) in do_start_loop'
10:25:29 jobs.1 | 	from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.3/lib/active_support/execution_wrapper.rb:92:in `wrap'
10:25:29 jobs.1 | 	from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/app_executor.rb:7:in `wrap_in_app_executor'
10:25:29 jobs.1 | 	from /Users/carlos/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/solid_queue-0.2.2/lib/solid_queue/processes/runnable.rb:47:in `block in do_start_loop'
@ArglosStudios
Copy link
Author

this happened both when running via the puma plugin or starting solid_queue via Procfile.

It still run jobs if I enqueue them. However the worker crashes often.

Workaround so far is to run inside docker

@natannobre
Copy link

natannobre commented Mar 27, 2024

I think I have the same issue on my mac. A piece of the message is solid_queue | /Users/natannobrechaves/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/pg-1.5.4/lib/pg/connection.rb:690: [BUG] Segmentation fault at 0x000000011d5b0b0e.

I'm running inside Procfile.dev.

@34code
Copy link

34code commented May 11, 2024

using rosetta to run your terminal (iterm for me) also seems to do the trick!

https://apple.stackexchange.com/questions/428768/on-apple-m1-with-rosetta-how-to-open-entire-terminal-iterm-in-x86-64-architec

@34code
Copy link

34code commented May 11, 2024

nvm.. pulled trigger too early. fails with rosetta as well

@feapaydin
Copy link

@natannobre I'm also having the same issue. Seems like it has been solved earlier here: #145 (comment)

@rosa
Copy link
Member

rosa commented Jul 29, 2024

Hey @ArglosStudios, is this still happening for you? Did #145 (comment) help at all?

@rjhancock
Copy link

I have this issue but with a MySQL backend (I don't have the option to swap it for another DB).

@rosa
Copy link
Member

rosa commented Aug 24, 2024

Hey @rjhancock, does this still happen for you? Are you running the last version?

@rjhancock
Copy link

Just tested with latest gems for everything and this is the error I get.

Screenshot 2024-08-24 at 15 31 07

@rosa
Copy link
Member

rosa commented Aug 24, 2024

Thanks a lot for testing 🙏
This one seems to be similar or the same as brianmario/mysql2#1163 🤔
It seems it affects Rails's parallel testing too: rails/rails#41176

There's a suggested solution here: compile mysql2 with a MySQL 5.7 client. Do you think this could work for you?

@rjhancock
Copy link

Sadly no as deployed MySQL is 8.0 same with local MySQL via Homebrew.

@rosa
Copy link
Member

rosa commented Aug 25, 2024

That's ok! You can continue using MySQL 8.0, this is just for the mysql2 gem. You can compile it against a MySQL 5.7 client and continue using it to access a MySQL 8.0 server.

This is what @johnmaxwell proposed here:

gem uninstall mysql2
brew install [email protected]
gem install mysql2 -- --with-mysql-dir="/usr/local/opt/[email protected]/"

@rjhancock
Copy link

When I test my Solid Queue code, I just run it within Docker Compose. Not ideal but works. I'd rather not add another monkey patch to the instructions and code base for a bug in one of two libraries that would bring my development environment more out of line from the production environment.

@rosa
Copy link
Member

rosa commented Aug 25, 2024

That makes sense! I wonder why you get the error then 🤔 In theory, this happens only in macOS, so I assumed you were running everything in your local machine.

@rjhancock
Copy link

Only happens on macOS. When running via Docker Compose, runs fine. The servers are all Ubuntu boxes running dockerized containers with Ruby 3.3.4 as the base image.

I try not to run the images inside Docker while working on the code as it greatly slows things down over all. I generally don't need to test the Job system as it's already well tested.

Most of the code I can run without working with queues so it doesn't impact too much. Just wish the bug could be found as looking through the solutions it looks like this has been a bug somewhere for the last several years.

@34code
Copy link

34code commented Aug 30, 2024

I recently switched to Omakub and ubuntu for development due to a number of macos related issues after the ARM on mac transition for this exact reason.. Like you, I run on production using kamal and docker containers (on x86_64 hw) so being on ubuntu helps me more accurately replicate production. Would be nice to identify a root cause on mac though.

@rosa
Copy link
Member

rosa commented Sep 13, 2024

Going to close this one as neither the segfault with PostgreSQL nor the file descriptor issue with MYSQL is a Solid Queue issue, but I hope both get fixed soon. In the meantime there are the workarounds mentioned here and in the issues linked.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants