Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure testing #929

Merged
merged 14 commits into from
Mar 22, 2018
Merged

Restructure testing #929

merged 14 commits into from
Mar 22, 2018

Conversation

jdpedrie
Copy link
Contributor

@jdpedrie jdpedrie commented Mar 9, 2018

This pull request updates the four test suites to be (i think -- see below for more) fully compatible with the restructure. It also improves the management of snippet tests especially.

Unit, snippet and conformance tests can be run as a whole or on a single repository. System tests can be run against the full project just as they are now.

Next up is the pre-release tests of each component as a separate unit, which will have the added benefit of proving that the unit, snippet and conformance tests work as expected as a discrete suite (as opposed to as part of the umbrella). That'll be done early next week.

I placed a couple of bootstrap files in the root of Core. I'm not especially happy with that, but they need to be available to every component, so the umbrella tests folder wasn't an option. They can be moved quite easily though.

sidenote: you don't need to include the composer autoloader. phpunit will find it on its own.

@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Mar 9, 2018
@michaelbausor
Copy link
Contributor

This looks really good!

Re the bootstrap files in the root of core - is it sensible to put them somewhere in Core/src/Testing? I am not sure whether that is better though.

@jdpedrie
Copy link
Contributor Author

jdpedrie commented Mar 15, 2018

@michaelbausor @dwsupplee Added the component integration.

A new CLI called dev/google-cloud integration handles it.

If a component is updated and about to be released, the test will use the local copy of that component. Otherwise it will allow composer to install at the requested version.

The process in detail:

  1. Copy each component into a folder called .testing.
  2. Check the local version of each component, then check the latest published version (API call to github). If the local version is higher than the published version (i.e. we're about to release that component), we'll add the component to a list.
  3. Each component is iterated over again. If that component requires or require-dev's a component in the first list (i.e. one which is about to be released), it'll be added to the list as well.
  4. Each component's composer.json file is modified. Any dependencies in the list have their version requirement set to @dev. All the components which are in the list are added as path repositories, so they are symlinked rather than installed from remote.
  5. Each component is iterated one last time, running composer install, then running unit and snippet tests. Any test failures or errors will stop execution.
  6. .testing is deleted.

If you run this locally, I suggest making sure you're authenticated with github by setting the $GH_OAUTH_TOKEN environment variable, otherwise you'll get rate limited soon enough.

The latest commit also includes some fixes to composer files to provide dependencies required by tests.

We'll need to figure out how to integrate this into our process. I'd like to setup some sort of a Travis trigger which will run against release pull requests. in the interim, it should be run locally before a release pull request is opened.

edit: it won't work out of the box until google/cloud-core is released with the new testing utilities. run dev/google-cloud release minor cloud-core locally first and it'll run.

@dwsupplee dwsupplee merged commit 06f02c5 into googleapis:restructure-apis Mar 22, 2018
dwsupplee pushed a commit to dwsupplee/gcloud-php that referenced this pull request Mar 23, 2018
* Add system test credentials CLI

* Update Snippet testing for restructure

* Fix unit tests for restructure

* Fix system test runner

* Fix conformance bootstrap

* fix credentials CLIts

* Fix docs

* exclude dev folder from snippet parsing

* install latest composer

* Credentials CLI should not throw exception

* Test component integration

* Don't fail if extension is missing

* skip tests if debugger ext is missing

* Allow setting the search location paths for the SourceLocationResolver (googleapis#6)
dwsupplee added a commit that referenced this pull request Mar 23, 2018
* Manual changes for restructure

* Restructure APIs via script

* Fix Debugger snippets

* Add github PR templates

* Initial fixes for subrepos

* Fix Core system tests

* Update BigQuery system tests

* Changes for Spanner

* Update phpunit files

* Update bootstrap files

* Datastore updates

* Add keyfile stub files

* Updates for Firestore

* Fix Language composer

* Fix logging tests and composer

* Fix pubsub composer and unit tests

* Update Speech composer

* Fix composer and tests for Storage

* Update Trace composer

* Update Translate composer

* Update Vision composer

* Update gapic only APIs

* Update composer files

* Rename test folders

* Move tests and update namespaces

* Update phpunit files

* Update Core

* Update doc generator to work with API restructure.

* Changes after merging from master

* Update after merge

* Small fixes for docs

* Update dev CLI for API restructure

* Updates after merge, update Snippet namespaces

* Move conformance and perf

* Update bootstrap files

* Update bootstrap

* Address PR comments

* Restore snippet test

* Remove keyfile stubs

* Update pubsub dependencies

* Update composer, move debugger bin

* Remove references to local Core

* Update bootstrap files

* Update docs

* Update core to pass doc gen

* Fix doc generator

* Add daemonutils to locate autoloader

* Remove DaemonUtils, update autoloader locator

* Restructure testing (#929)

* Add system test credentials CLI

* Update Snippet testing for restructure

* Fix unit tests for restructure

* Fix system test runner

* Fix conformance bootstrap

* fix credentials CLIts

* Fix docs

* exclude dev folder from snippet parsing

* install latest composer

* Credentials CLI should not throw exception

* Test component integration

* Don't fail if extension is missing

* skip tests if debugger ext is missing

* Allow setting the search location paths for the SourceLocationResolver (#6)

* update bin paths

* move dlp v2 to new structure

* require opis/closure for tests

* add internal tags and fix cs

* update configurations

* resolve issues with perf/conformance tests

* update bootstraps

* update PR template

* show bigtable on docs

* fix doc generation
@jdpedrie jdpedrie deleted the restructure-testing branch September 20, 2018 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants