Test suite for non-core plugins

[drewbanin]

Describe the change

Let's add a test suite for the dbt-spark plugin. This plugin should be built with the goal of extension for all dbt plugins in the future, but the first cut should be spark-specific to make the scope manageable.

Test suite goals:

• assert compliance for dbt-spark with core functionality
• help plugin maintainers upgrade to new versions of Core

Describe alternatives you've considered

We could make per-plugin integration test suites. This would lead to a ton of repeated work for community-contributed plugin authors.

Additional context

We've discussed at length some ideas around the implementation of this test suite, but the final implementation remains a little bit nebulous. There are lots of ways for us to make our first forays here. I think for a first cut, we should:

• Create generic-ish integration tests for core dbt functionality (creating tables, views, incrementals, etc)
• Allow plugins to define which dbt features are unsupported, and skip related tests or allow them to fail
• Allow extension of tests
  • This is an important one. Spark's vanilla incremental models require the specification of a partition_by config to run. Spark's integration tests should be able to assert that incremental models work correctly given the required configurations for these models.

Who will this benefit?

• dbt maintainers
• plugin maintainers
• consumers of plugins (which features are supported?)

[jtcohen6]

I had incorrectly caused this issue to be closed by linking it to #2610, which was a prerequisite and not a resolution.

I am, however, going to close older issues that relate to refactoring the test suite / dbt-integration-tests, with the shared goal of easing the path for plugin development.