Allow running acceptance tests on travis #230

3flex · 2016-09-24T22:48:16Z

Closes #80

Full credit to Puppet from where this was lifted (puppetlabs/modulesync_configs#58), with a couple of minor changes to ensure it fits better with the Voxpupuli Gemfile.

Tested with puppet-nginx module.

The biggest change compared with Puppet's version is that acceptance tests are NOT run by default. Each module would have to enable docker sets in their Travis file (see https://github.com/voxpupuli/puppet-nginx/pull/885/files#diff-f620401c6bdf661e1275316a6e01b62f for an example).

It's probably a good idea to enable by default in future but I thought it might be too disruptive right now.

wyardley · 2016-09-26T18:08:46Z

Wow, thanks @3flex...

bastelfreak · 2016-09-29T08:23:31Z

I think we should not provide default_sets in our default config for all modules. Most of them don't have working acceptance tests.

In general I'm against acceptance tests on travis. This slows down the matrix. We currently have 10 repos and only get resources on a per org basis, not per repo. And our single testmatrix already takes such a long time. We are already discussing it in #80, maybe we should continue there before we merge/decline this one?

juniorsysadmin · 2017-01-01T06:40:06Z

There are a bunch of docker nodesets in this repo now. Can this Pull Request be closed?

3flex · 2017-01-01T19:41:40Z

@juniorsysadmin this PR was less about adding docker nodesets and more about enabling support for running acceptance tests on Travis.

If this were to be merged I can easily rebase and remove the nodesets the PR originally introduced.

juniorsysadmin · 2017-01-03T11:10:20Z

@3flex

If this were to be merged I can easily rebase and remove the nodesets the PR originally introduced.

Go ahead

3flex · 2017-01-05T03:23:53Z

Rebased, and made a couple of minor adjustments for clarity.

Though there wasn't a final consensus in #80, I'm not sure there's a good reason not to merge this... as the changes in puppet-corosync proved, this PR isn't required to enable running acceptance tests on Travis, it just makes it a little simpler to do, and the functionality it enables is entirely opt-in on a per-module basis.

I tested against puppet-nginx and puppet-corosync for sanity, since nginx module has some docker_sets in its .sync.yml, and corosync has a more complex config file.

Diff from nginx module:

diff --git a/.msync.yml b/.msync.yml
index d505802..1bc592b 100644
--- a/.msync.yml
+++ b/.msync.yml
@@ -1 +1 @@
-modulesync_config_version: '0.16.7'
+modulesync_config_version: '0.16.8'
diff --git a/.travis.yml b/.travis.yml
index da95439..61b4de3 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,7 +2,6 @@
 sudo: false
 language: ruby
 cache: bundler
-bundler_args: --without system_tests development
 before_install:
   - bundle -v
   - rm Gemfile.lock || true
@@ -15,23 +14,44 @@ script:
 matrix:
   fast_finish: true
   include:
+  - rvm: 2.1.9
+    bundler_args: --without development
+    dist: trusty
+    env: PUPPET_INSTALL_TYPE=agent BEAKER_debug=true BEAKER_set=docker/ubuntu-14.04 CHECK=beaker
+    services: docker
+    sudo: required
+  - rvm: 2.1.9
+    bundler_args: --without development
+    dist: trusty
+    env: PUPPET_INSTALL_TYPE=agent BEAKER_debug=true BEAKER_set=docker/centos-7 CHECK=beaker
+    services: docker
+    sudo: required
   - rvm: 1.9.3
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 3.0" STRICT_VARIABLES="yes" CHECK=test
   - rvm: 1.9.3
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 3.0" STRICT_VARIABLES="yes" FUTURE_PARSER="yes" CHECK=test
   - rvm: 2.1.9
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 3.0" STRICT_VARIABLES="yes" CHECK=test
   - rvm: 2.1.9
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=test
   - rvm: 2.2.6
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=test
   - rvm: 2.3.3
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=build DEPLOY_TO_FORGE=yes
   - rvm: 2.3.3
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=rubocop
   - rvm: 2.3.3
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=test
   - rvm: 2.4.0
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=test
   allow_failures:
     - rvm: 2.4.0

Diff from corosync module:

diff --git a/.msync.yml b/.msync.yml
index d505802..1bc592b 100644
--- a/.msync.yml
+++ b/.msync.yml
@@ -1 +1 @@
-modulesync_config_version: '0.16.7'
+modulesync_config_version: '0.16.8'
diff --git a/.travis.yml b/.travis.yml
index 007d519..a9b6e6b 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -2,7 +2,6 @@
 sudo: false
 language: ruby
 cache: bundler
-bundler_args: --without system_tests development
 before_install:
   - bundle -v
   - rm Gemfile.lock || true
@@ -16,43 +15,52 @@ matrix:
   fast_finish: true
   include:
   - rvm: 1.9.3
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 3.0" STRICT_VARIABLES="yes" CHECK=test
   - rvm: 1.9.3
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 3.0" STRICT_VARIABLES="yes" FUTURE_PARSER="yes" CHECK=test
   - rvm: 2.1.9
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 3.0" STRICT_VARIABLES="yes" CHECK=test
   - rvm: 2.1.9
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=test
   - rvm: 2.2.6
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=test
   - rvm: 2.3.3
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=build DEPLOY_TO_FORGE=yes
   - rvm: 2.3.3
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=rubocop
   - rvm: 2.3.3
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=test
   - rvm: 2.4.0
+    bundler_args: --without system_tests development
     env: PUPPET_VERSION="~> 4.0" CHECK=test
   - rvm: 2.3.1
+    bundler_args: --without development
     env: PUPPET_INSTALL_TYPE=agent CHECK=acceptance BEAKER_debug=true BEAKER_set=docker/centos-7
     services: docker
     sudo: required
-    bundler_args: --without development
   - rvm: 2.3.1
+    bundler_args: --without development
     env: PUPPET_INSTALL_TYPE=agent CHECK=acceptance BEAKER_debug=true BEAKER_set=docker/debian-8
     services: docker
     sudo: required
-    bundler_args: --without development
   - rvm: 2.3.1
+    bundler_args: --without development
     env: PUPPET_INSTALL_TYPE=agent CHECK=acceptance BEAKER_debug=true BEAKER_set=docker/ubuntu-14.04
     services: docker
     sudo: required
-    bundler_args: --without development
   - rvm: 2.3.1
+    bundler_args: --without development
     env: PUPPET_INSTALL_TYPE=agent CHECK=acceptance BEAKER_debug=true BEAKER_set=docker/ubuntu-16.04
     services: docker
     sudo: required
-    bundler_args: --without development
   allow_failures:
     - rvm: 2.4.0
 branches:

sacres · 2017-01-05T15:21:30Z

I recently added acceptance to puppet-jira for docker on travis; I'm on the positive side of this.

rnelson0 · 2017-01-05T15:43:30Z

This is going to need a rebase however, as we just dropped puppet3 support in the configs. It's pretty lengthy so I don't trust myself to resolve the conflicts on my own esp since I can't test it.

3flex · 2017-01-05T16:30:13Z

Just rebased using Github's web UI, so please do a squash and merge if this is to be merged as it added another commit.

3flex · 2017-03-27T17:47:44Z

Is there any other action required for this to be merged? Happy to make any changes, just don't want the PR to get stale again!

vinzent

Running acceptance tests automatically is IMHO great!

Opt-in is also cool because for example puppet-selinux won't work in docker.

vinzent · 2017-04-15T09:25:41Z

test PR on puppet-nginx showed that beaker tests were run. test on puppet-selinux showed that beaker tests were not added to travis.yml (because not configured to do).

This was referenced Sep 24, 2016

Configure acceptance tests on docker on travis voxpupuli/puppet-nginx#885

Merged

add Ubuntu 16.04 as default nodeset #229

Closed

alexjfisher mentioned this pull request Sep 25, 2016

Pipe was broken in the previous rubucop fix voxpupuli/puppet-php#243

Merged

bastelfreak mentioned this pull request Sep 29, 2016

Fix for issue #552 including acceptance tests voxpupuli/puppet-collectd#556

Merged

bastelfreak added the needs-feedback label Sep 29, 2016

dhoppe mentioned this pull request Oct 13, 2016

Add support for additional parameters #241

Merged

juniorsysadmin added needs-rebase needs-work and removed needs-feedback labels Jan 3, 2017

3flex force-pushed the puppet_on_docker_on_travis branch from 40ee40b to d93302e Compare January 5, 2017 02:51

3flex added enhancement and removed needs-rebase needs-work labels Jan 5, 2017

3flex mentioned this pull request Jan 5, 2017

Add new .travis.yml section for beaker-docker tests #80

Closed

juniorsysadmin added the needs-feedback label Jan 5, 2017

juniorsysadmin approved these changes Jan 5, 2017

View reviewed changes

vinzent approved these changes Apr 11, 2017

View reviewed changes

allow running acceptance tests on travis

a61ef39

vinzent force-pushed the puppet_on_docker_on_travis branch from 13661bf to a61ef39 Compare April 15, 2017 08:48

vinzent mentioned this pull request Apr 15, 2017

Test modulesync: allow running acceptance tests on travis #230 voxpupuli/puppet-nginx#1078

Closed

vinzent changed the title ~~allow running acceptance tests on travis~~ Allow running acceptance tests on travis Apr 15, 2017

vinzent removed the needs-feedback label Apr 15, 2017

vinzent merged commit 4cd88a6 into master Apr 15, 2017

bastelfreak deleted the puppet_on_docker_on_travis branch April 15, 2017 12:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow running acceptance tests on travis #230

Allow running acceptance tests on travis #230

3flex commented Sep 24, 2016 •

edited

Loading

wyardley commented Sep 26, 2016

bastelfreak commented Sep 29, 2016

juniorsysadmin commented Jan 1, 2017

3flex commented Jan 1, 2017

juniorsysadmin commented Jan 3, 2017

3flex commented Jan 5, 2017

sacres commented Jan 5, 2017

rnelson0 commented Jan 5, 2017

3flex commented Jan 5, 2017

3flex commented Mar 27, 2017

vinzent left a comment

vinzent commented Apr 15, 2017

Allow running acceptance tests on travis #230

Allow running acceptance tests on travis #230

Conversation

3flex commented Sep 24, 2016 • edited Loading

wyardley commented Sep 26, 2016

bastelfreak commented Sep 29, 2016

juniorsysadmin commented Jan 1, 2017

3flex commented Jan 1, 2017

juniorsysadmin commented Jan 3, 2017

3flex commented Jan 5, 2017

Diff from nginx module:

Diff from corosync module:

sacres commented Jan 5, 2017

rnelson0 commented Jan 5, 2017

3flex commented Jan 5, 2017

3flex commented Mar 27, 2017

vinzent left a comment

Choose a reason for hiding this comment

vinzent commented Apr 15, 2017

3flex commented Sep 24, 2016 •

edited

Loading