Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to Puma #8

Merged
merged 32 commits into from
May 20, 2013
Merged

Move to Puma #8

merged 32 commits into from
May 20, 2013

Conversation

sethvargo
Copy link
Contributor

I went a little hackyety-hack-hack tonight.

  1. Move to puma for concurrency and JRuby support
  2. Depend on chef-pedant as a library
  3. Minor tweaks to options
  4. Create a single default options hash
  5. Put it on travis for testing

@sethvargo
Copy link
Contributor Author

Things I don't understand:

@josephholsten
Copy link

I don't know why Mixlib::ShellOut isn't working on JRuby today, but it used to be quite a pain to manage normal fork + exec semantics from in the JVM. JRuby went through a bit of shenanigans to get Kernel#system working and it still isn't completely safe.

It should be possibly to get it working via posix_spawn(3), but that is work that needs to be done.

Alternatively, JRuby 1.7 release notes say that Kernel#exec now works out of the box. It doesn't have all the semantics we want from Mixlib::ShellOut, but it's implementation probably a good starting point to build on.

@sethvargo
Copy link
Contributor Author

It uses fork. Fork doesn't work on JRuby. It's not an easy as a change as you think - the dependency is coming from chef-pedant.

@erikh
Copy link

erikh commented May 17, 2013

Process.spawn on 1.9.2+ implements posix_spawn() -- might be an option here since I'm pretty sure JRuby does it as a part of its 1.9 support. I also know popen works on jruby.

@josephholsten
Copy link

Yeah, definately a sticky problem. It's easy to run the proc via posix_spawn(3), it's just that Mixlib::ShellOut does so much to exec cleanly.

Here's the two uses in chef-pedant:

The ruby-related environment var blanking implies these both really depend on shell_out's cleanliness.

@erikh
Copy link

erikh commented May 17, 2013

wow, edited that comment, seems the formatting was messed up from emailing it. tl;dr -- Process.spawn can help and likely can replace the need for Shellout.

here's the docs: http://ruby-doc.org/core-1.9.3/Process.html#method-c-spawn

@sethvargo
Copy link
Contributor Author

@erikh @josephholsten right, but I'm not sure chef-pedant is accepting PRs. We use it internally to test things, so somethings are "just the way they are".

@jkeiser
Copy link
Contributor

jkeiser commented May 18, 2013

I think chef-pedant will take PRs. Need to run against chef 11 server first though.

@erikh
Copy link

erikh commented May 18, 2013

Might be better to do this to mixlib-shellout instead -- depends on your need as a company for ruby 1.8 still. No time for it myself (sorry!), but if you look at the docs it shouldn't be too complicated to make shellout play nicely with Process.spawn.

@sethvargo
Copy link
Contributor Author

@jkeiser are these legitimate test failures on chef-zero? Or something related to the move to puma/changing pedant?

This message was sent from my mobile device.

I apologize in advance for any typographical errors or autocorrections.

On May 17, 2013, at 10:14 PM, Erik Hollensbe [email protected] wrote:

Might be better to do this to mixlib-shellout instead -- depends on your need as a company for ruby 1.8 still. No time for it myself (sorry!), but if you look at the docs it shouldn't be too complicated to make shellout play nicely with Process.spawn.


Reply to this email directly or view it on GitHub.

@jkeiser
Copy link
Contributor

jkeiser commented May 19, 2013

Not sure--almost certainly some new pedant tests that I haven't looked at yet. I know there are something like 45 failures right now without any patches (Jamie noticed this).

@sethvargo
Copy link
Contributor Author

@jkeiser okay. That's similar to what I'm seeing.

Also, the change to puma marks a significant unintended side-effect. CZ was already really fast, but, for some reason (I'm not sure), Puma makes it even faster. I was using it against the berkshelf test suite and the entire runtime decreased by 1 min.

@sethvargo
Copy link
Contributor Author

@jkeiser I just rebased with master and I'm still seeing failures. I also tried master and I'm seeing failures.

@generate_real_keys = options[:generate_real_keys]

ChefZero::Log.level = options[:debug] ? :debug : :info
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like it should be set outside of ChefZero::Server, so that you can set it to :warning or :error if you want.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a780cbb, I changed it to accept a -l flag instead of -d, so the user can set the value via the CLI. The user can already manipulate this value programmatically the same way I've done here.

@jkeiser
Copy link
Contributor

jkeiser commented May 19, 2013

@sethvargo the patch looks great in general, just a few things. Check run-pedant.rb: it should be doing a reset --hard to a specific git tag. Can you check if your "rake test" / "rake spec" is doing that?

@sethvargo
Copy link
Contributor Author

@jkeiser I just did that in a2dd6c4 via the Gemfile (since we aren't doing a system exec). Bundler takes care of it.

@sethvargo
Copy link
Contributor Author

@jkeiser I'm not sure why there's these three failures. I don't think it's related to the move to puma:

module ChefZero
autoload :Log, 'chef_zero/log'
require 'chef_zero/core_ext'
require 'chef_zero/log'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does requiring these inside ChefZero do? Why get rid of the autoload?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requiring them inside guarantees the module is already created. Plus it keeps all the CZ requires in place.

Kernel#autoload is not threadsafe. Puma is multi-threaded.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it matter if the module is already created? It'll get created by whichever require opens it first. This is not an expected place to see a require, so if we don't have to do it this way for a specific reason, I'd love it to be at the top.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does if you do this:

class ChefZero::Foo

end

in one of those future requires. It's actually a very common pattern.

@sethvargo
Copy link
Contributor Author

@jkeiser okay, finally it's just the moneta issue.

jkeiser added a commit that referenced this pull request May 20, 2013
@jkeiser jkeiser merged commit 0f31259 into chef:master May 20, 2013
@chef chef locked and limited conversation to collaborators Nov 16, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants