Allow build jobs to be configured and managed by puppet. Includes #163 a...

[jchristi]

...nd resolves #120.

[jenkinsadmin]

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

[jchristi]

This PR adds:

• A define jenkins::job that takes a template for a jenkins job's config.xml file and either installs, updates or removes the job. There is also an option to have the job enabled or disabled.
• Added two refresh-only exec's to jenkins::cli: one for reloading configuration from disk and one for doing a safe restart of jenkins. These are needed for Jenkins to recognize when a job is updated.
• a new parameter on the main jenkins class: job_hash which behaves the same way as config_hash and plugin_hash and has an associated class jenkins::jobs.
• Added and updated spec tests for the new defines and classes
• Updated README.md
• Updated test manifests using the jenkins::job define
• Updated contrib/examples with an example usage of jenkins::job

Notes:

• The two new exec's in jenkins::cli have a high retry count set because in my usage I found they don't always complete on the first try, but with a retry count I haven't had any issues.
• The jenkins::job define subsequently calls jenkins::job::present or jenkins::job::absent based on the value of the ensure parameter. This is to keep the code cleaner since jenkins::job::present is somewhat large.
• The jenkins::job::present define has some odd looking code in it. Ideally, we would like to just be able to lay down the config.xml file into the appropriate /var/lib/jenkins/jobs/<name> directory and Jenkins would recognize that and be happy. Unfortunately, Jenkins will not recognize new jobs unless you tell it in a very specific way and the only way I've found that works is by using the jenkins cli's create-job command which requires reading the config.xml from stdin. It is pretty annoying but it works.
• If the above point weren't enough, Jenkins makes life further more difficult by applying its own re-formatting to your config.xml, which makes job management via puppet not idempotent. To rectify this, the jenkins::job::present define applies some regex substitutions to your job's config.xml.erb template to match Jenkin's formatting behavior before laying down the file in /tmp so a that a diff between the two files can be done to determine if a change is needed and thus achieving idempotence. I suspect the duplication of config files in /tmp will be controversial, but fortunately the management of jobs is an opt-in feature.

[rtyler]

Holy smokes this is a big PR

[jchristi]

Hence the write-up. Half or more of the code is test code though

[jchristi]

The best place to start would be manifests/job.pp. Everything revolves around that.

[jchristi]

@rtyler it looks like there has been a lot of activity going on with this project this week. I would really like to get this major feature enhancement added to your next release. Please let me know if there is anything more I can do to get this merged. I'd be happy to schedule a call or discuss on IRC.

[rtyler]

@jchristi would it be possible to rebase this change on top of master? I'm not opposed to pulling this in but I won't have a huge amount of time to resolve conflicts and test :(

[jchristi]

@rtyler rebased. There appear to be build issues unrelated to this PR though.

[rtyler]

I'm going to rebase this one more time and do some local testing.

[rtyler]

@jchristi It looks like this fails some of the serverspec, I think there's some resource dependency stuff that's busted.

[15:54:49] tyler:puppet-jenkins git:(master*) $ bundle exec vagrant provision        
Vagrant appears to be running in a Bundler environment. Your 
existing Gemfile will be used. Vagrant will not auto-load any plugins
installed with `vagrant plugin`. Vagrant will autoload any plugins in
the 'plugins' group in your Gemfile. You can force Vagrant to take over
with VAGRANT_FORCE_BUNDLER.

/usr/home/tyler/.rvm/gems/ruby-2.1.0@rubygems/bundler/gems/vagrant-c38e17cc29f6/lib/vagrant/shared_helpers.rb:13: warning: already initialized constant Vagrant::DEFAULT_SERVER_URL
/home/tyler/.rvm/gems/ruby-2.1.0@rubygems/bundler/gems/vagrant-c38e17cc29f6/lib/vagrant/shared_helpers.rb:13: warning: previous definition of DEFAULT_SERVER_URL was here
/usr/home/tyler/.rvm/gems/ruby-2.1.0@rubygems/bundler/gems/vagrant-c38e17cc29f6/lib/vagrant/version.rb:5: warning: already initialized constant Vagrant::VERSION
/home/tyler/.rvm/gems/ruby-2.1.0@rubygems/bundler/gems/vagrant-c38e17cc29f6/lib/vagrant/version.rb:5: warning: previous definition of VERSION was here
You appear to be running Vagrant outside of the official installers.
Note that the installers are what ensure that Vagrant has all required
dependencies, and Vagrant assumes that these dependencies exist. By
running outside of the installer environment, Vagrant may not function
properly. To remove this warning, install Vagrant using one of the
official packages from vagrantup.com.

==> ubuntu-precise: Rsyncing folder: /usr/home/tyler/source/github/puppet/puppet-jenkins/ => /vagrant/jenkins
==> ubuntu-precise: Rsyncing folder: /usr/home/tyler/source/github/puppet/puppet-jenkins/ => /vagrant
==> ubuntu-precise: Rsyncing folder: /usr/home/tyler/source/github/puppet/puppet-jenkins/spec/serverspec/ubuntu-precise/manifests/ => /tmp/vagrant-puppet-2/manifests
==> ubuntu-precise: Rsyncing folder: /usr/home/tyler/source/github/puppet/puppet-jenkins/ => /tmp/vagrant-puppet-2/modules-0
==> ubuntu-precise: Rsyncing folder: /usr/home/tyler/source/github/puppet/puppet-jenkins/spec/fixtures/modules/ => /tmp/vagrant-puppet-2/modules-1
==> ubuntu-precise: Running provisioner: shell...
    ubuntu-precise: Running: inline script
==> ubuntu-precise: sudo
==> ubuntu-precise: : 
==> ubuntu-precise: unable to resolve host vagrant-jenkins-ubuntu-precise
==> ubuntu-precise: stdin: is not a tty
==> ubuntu-precise: Running provisioner: shell...
    ubuntu-precise: Running: inline script
==> ubuntu-precise: sudo
==> ubuntu-precise: : 
==> ubuntu-precise: unable to resolve host vagrant-jenkins-ubuntu-precise
==> ubuntu-precise: stdin: is not a tty
==> ubuntu-precise: Running provisioner: puppet...
==> ubuntu-precise: Running Puppet with default.pp...
==> ubuntu-precise: sudo
==> ubuntu-precise: : 
==> ubuntu-precise: unable to resolve host vagrant-jenkins-ubuntu-precise
==> ubuntu-precise: stdin: is not a tty
==> ubuntu-precise: Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaults
==> ubuntu-precise: Notice: Scope(Node[default]): Hello world from vagrant-jenkins-ubuntu-precise}
==> ubuntu-precise: Notice: Compiled catalog for vagrant-jenkins-ubuntu-precise.us-west-2.compute.internal in environment production in 1.34 seconds
==> ubuntu-precise: Error: Validation of Exec[reload-jenkins] failed: 'java -jar /usr/share/jenkins/jenkins-cli.jar -s http://localhost:8080 reload-configuration' is not qualified and no path was specified. Please qualify the command or specify a path. at /tmp/vagrant-puppet-2/modules-0/jenkins/manifests/cli.pp:48
==> ubuntu-precise: Wrapped exception:
==> ubuntu-precise: 'java -jar /usr/share/jenkins/jenkins-cli.jar -s http://localhost:8080 reload-configuration' is not qualified and no path was specified. Please qualify the command or specify a path.
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.
[16:00:20] tyler:puppet-jenkins git:(pr/169*) $ 

[jchristi]

Error: Validation of Exec[reload-jenkins] failed: 'java -jar /usr/share/jenkins/jenkins-cli.jar -s http://localhost:8080 reload-configuration' is not qualified and no path was specified. Please qualify the command or specify a path

What does that even mean?

[rtyler]

@jchristi this is when provisioning an EC2 machine (bundle exec vagrant up in HEAD). The use of this isn't terribly well documented right now, I'll work on that.

It looks like an Exec[] is not being guarded with a refreshonly or a require set up properly

[jchristi]

@rtyler are you sure you rebased correctly? refresonly and require are definitely in place: https://github.com/jenkinsci/puppet-jenkins/pull/169/files#diff-51306f26e2f26dcf062139b157b7f3ffR46

[jchristi]

Rebased. The only conflict was in init.pp so its hard to imagine it was a rebase issue, but the error is at odds with whats in the branch. Any additional info would be useful. I assume this PR is not getting into your release...

[jchristi]

Well the spec tests aren't passing anymore. I don't know what has been added, but I'll take a look at it tomorrow.

[rtyler]

I'm tired of lookin' at this PR, if it breaks tests in master I'll just deal with that this weekend

[bracki]

Does not work:

err: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find resource 'Class[Jenkins::Jobs]' for relationship from 'Class[Jenkins::Service]' on node blahblabhalbha.eu-west-1.aws.jimdo-server.com
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run

jenkins::jobs is never included in init.pp.

[jchristi]

@bracki, that appears to be the case. init.pp was refactored and that must have been missed during rebase or merge. I'll open a new MR.