VAGOV-2142: Add yaml-tests to include phpunit, phpcs, and linting

[jonpugh]

This PR includes a new library I created to make maintaining and running a list of tests really easy. It also integrates directly with GitHub so each test is passed back to the PR as a commit status.

Simply composer install

Then composer yaml-tests or just composer y

Thew way yaml tests work, if there is a GITHUB_TOKEN environment variable, it will not only run the tests but post to the commit status API with the test pass or fail.

[jonpugh]

So, this PR is the test itself...

The yaml-tests command can be run from anywhere, as long as it has a GITHUB_TOKEN it can post commit status back here...

From my local laptop I am about to run composer y --github-token=exxxx

[jonpugh]

The tests are defined in a tests.yml file:

va/phpunit:
  description: PHPUnit
  command: phpunit tests/phpunit --colors=always
va/phpcs:
    - phpcs --standard=PSR2 -n docroot/modules/custom docroot/themes --colors
va/phplint:
    - find docroot/modules/custom docroot/themes -name '*.php' -print0 | xargs -0 -n1 php -l
    - find docroot/modules/custom docroot/themes -name '*.module' -print0 | xargs -0 -n1 php -l
    - find docroot/modules/custom docroot/themes -name '*.install' -print0 | xargs -0 -n1 php -l

[va-cms-bot]

Created an environment test-vagovcms#pr-348

[jonpugh]

The plugin includes an option for --test-file so we can specify a totally separate file for production tests.

[ElijahLynn]

Nice work @jonpugh, this is highly useful and powerful giving engineers the ability to control which tests are run (and on a PR level granularity) and love the status feedback here on the PR, so much more useful than a global pass/fail.

[va-cms-bot]

Created an environment test-vagovcms#pr-348

[va-cms-bot]

Created an environment test-vagovcms#pr-348

[jonpugh]

Ok!

I've got PHPCS and Linting now working!

Good new on another front: I swapped out the pre-commit hook to just call composer va:test:cs, so now the automated tests from yaml-test and the git commit pre-hook are running exactly the same command.

The pre-commit git hook file had additional logic to skip files, and other things. That logic is now in the composer command va:test:cs, including ignoring certain files.

[va-cms-bot]

Created an environment devops-vagovcms#pr-348

[ElijahLynn]

W00t w00t!

[jonpugh]

👍👏👏👏

…

On Fri, Jun 7, 2019, 1:16 AM Elijah Lynn ***@***.***> wrote: W00t w00t! [image: image] <https://user-images.githubusercontent.com/1504756/59082588-b4c1e500-88a8-11e9-929f-4aa243d6a87f.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#348?email_source=notifications&email_token=AAAZ7NEX673HO34LLJ3VC4DPZHVKNA5CNFSM4HPSUEIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXE3TAY#issuecomment-499759491>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAZ7NGNRXXIFR6BFFTXVBLPZHVKNANCNFSM4HPSUEIA> .

[jonpugh]

Incredible!
Thanks so much for taking this over the line, @ElijahLynn . The last few problems were DevOps side, it was hard for me to debug workout being now familiar.

[jonpugh]

Incredible!
Thanks so much for taking this over the line, @ElijahLynn . The last few problems were DevOps side, it was hard for me to debug workout being now familiar.

[jonpugh]

The last steps as I see it:

• Add --hostname= option back to the script. That will change the "aa21xxx" to be the url of the environment.
• merge DevOps repo branch.
• change repo ref back to master. Push, wait for another PASS
• once pass happens on master DevOps repo, we should tag a release of that and put the tag in VAGOV___GIT_REF variable.

[ElijahLynn]

The last steps as I see it:

• Add --hostname= option back to the script. That will change the "aa21xxx" to be the url of the environment.

I don't want to hold up this PR because of the hostname. And we can't just add it back as it broke docker last night because it was too long, so we need more time to figure out how to add a hostname longer than 64 chars, unless you already know how. Do you see this as a blocker for merge?

• merge DevOps repo branch.
• change repo ref back to master. Push, wait for another PASS
• once pass happens on master DevOps repo, we should tag a release of that and put the tag in VAGOV___GIT_REF variable.

[jonpugh]

rebuild

[ElijahLynn]

@beeyayjay The MigrationCount failed just now. Should that test be disabled or kept up to date?
05317f2

[ElijahLynn]

rebuild

[va-cms-bot]

Created an environment devops-vagovcms#pr-348

[va-cms-bot]

Created an environment devops-vagovcms#pr-348

[ElijahLynn]

failed on performance test, I think there was something up with Jenkins slave container host on that one. The time to post status here was really long after the Ansible task started.

[ElijahLynn]

build

[ElijahLynn]

failed on performance test, I think there was something up with Jenkins slave container host on that one. The time to post status here was really long after the Ansible task started.

Almost feel like before any performance test, there needs to be a system check for CPU and Disk I/O that isn't dependent on the app and just tests the raw system.

[ElijahLynn]

failed on performance test, I think there was something up with Jenkins slave container host on that one. The time to post status here was really long after the Ansible task started.

Come to think of it, the Jenkins master was displaying a "Jenkins is going to shutdown" message due to a scheduled restart so I do wonder if that had something to do with it.

[va-cms-bot]

Created an environment test-vagovcms#pr-348

[ElijahLynn]

Huzzah! Merging, GREAT WORK @jonpugh! 🚀 🎉 🎸