Default to installing the same Chef version on the node as is used on work station

[tmatilai]

knife bootstrap works like this, and it offers less surprises like currently when Chef 11 is released. This is related to #185 as the same logic would be good for both omnibus and gem installers.

[dsyer]

I vote for this as the default. Chef 11 seems to have some incompatibilities with ec2 and knife plugins, and it's not the first time this has happened when chef is revved. I worked quite hard to get a set of gems installed locally that worked, and then "knife prepare" went and installed different (broken) ones on the remote server. Error output:

$ knife cook -i ubuntu@$AWSHOST
ERROR: RuntimeError: The chef gem on ec2....com is out of date. Please run `knife prepare [email protected]` to upgrade Chef to ~>0.10.4.

[matschaffer]

@dsyer agreed. But your error message makes me wonder if you're using the right version of knife-solo. knife cook should throw a deprecation warning in the latest release. You may want to run gem clean knife-solo and try again.

[tmatilai]

@dsyer Yeah that error message comes from your local machine. knife cook has been renamed to knife solo cook already in v0.1.0 and it warns about deprecation now. So you seem to have (also) knife-solo v0.0.x installed and Knife is loading that. Make sure you have the latest version (0.2.0) installed and remove the old ones using the command @matschaffer gave.

Also ec2 and other knife plugins won't be installed on the node. So you can use Chef 11 on the node independent of the version on your working station. But incompatibility issues might rise from cookbooks. Anyway you can force the version on the node by giving e.g. --omnibus-version=10.22.0 option to knife solo bootstrap or knife solo prepare.

But I will anyway fix this issue soon unless someone is faster. =)

[tmatilai]

@dsyer were you able to resolve your problem?

[dsyer]

Didn't have time yesterday, but thanks for following up. I will try --omnibus-version=... when I get a chance. I have to say it was pretty obscure, if that turns out to be the workaround.

I can confirm that I was using an earlier version of knife solo. Here's a section of my Gemfile.lock

    knife-ec2 (0.5.10)
      fog (~> 0.8.2)
    knife-solo (0.0.8)
      chef (~> 0.10.0)
      net-ssh (~> 2.1.3)

This was the only combination of chef, knife, knife-solo, knife-ec2 and the net-ssh gems I could find that worked (coped a Gemfile from a helpful old blog). If it doesn't work out of the box without specifying explicit dependencies, then I think it's a big pit of potentially wasted time. Is there anything I can do to get a reliable set of modern dependencies? A Gemfile that someone could share for instance?

[tmatilai]

The --omnibus-version switch will be renamed to --bootstrap-version to be compatible with knife bootstrap, knife ec2 server create, etc. It will then also work with gem installer, not only with omnibus like now. Hopefully we can merge that (#185) to master even today and I can then start working with this issue.

I tested quickly with a Gemfile like this:

source "https://rubygems.org"

gem 'chef', '~> 10'
gem 'knife-ec2'
gem 'knife-solo'

Bundler seemed to resolve all of them to the latest versions (knife-ec2 v0.6.2, knife-solo v0.2.0, chef v10.22.0). The same with Chef v11.4.0. There has been a lot of dependency based problems lately in Chef, so it could have been different story some days ago. bundle exec knife ec2 server create seemed to work just fine with Chef 11.

[dsyer]

I never had a problem with knife ec2 create (did I say I did?), unless you count the error at the end which seems to be ignorable (ERROR: Errno::ENOENT: No such file or directory - /etc/chef/validation.pem). It was knife cook that wasn't working. It still wasn't when I tried it again today, although if I understand correctly I should get a deprecation warning? I didn't:

$ knife cook ubuntu@$AWSHOST
ERROR: knife encountered an unexpected error
This may be a bug in the 'cook' knife command or plugin
Please collect the output of this command with the `-VV` option before filing a bug report.
Exception: NoMethodError: super: no superclass method `run' for #<Chef::Knife::Cook:0x00000000e432c0>

It turned out I had forgotten to run knife prepare (knife solo prepare). So maybe that's a bug (the error message is not very clear about the mistake)? It did work eventually, in the sense that I got no error output (my cookbooks were still not executed - probably a mistake on my part), but I have no idea if that's because something else changed or because I upgraded knife. This is from my new Gemfile.lock:

    knife-ec2 (0.6.2)
      chef (>= 0.10.10)
      fog (~> 1.6)
    knife-solo (0.2.0)
      chef (>= 10.12)
      librarian (~> 0.0.20)
      net-ssh (>= 2.2.2, < 3.0)

[tmatilai]

The error message still gives a hint you have (also) some old knife-solo gem installed. Try gem clean knife-solo. You also might want to run the knife commands with bundle exec, but due to a Knife bug it doesn't help always with Knife plugins...

[tmatilai]

I'll boldly merge this. @matschaffer can fix it later if needed. ;)