[CI/CD Improvements] Migrate to hatch and standardize workflows

[mikealfare]

Short description

As a maintainer, I want to standardize and modernize development tooling, so that I can reduce maintenance issues and focus more on content than form.

Objectives

• Standardize workflows across all adapters; use dbt-postgres as the pilot
• Minimize dependencies between workflows
  • Move shared dependencies into actions
• Reduce GHA footprint in dbt-spark
  • Move repo-agnostic actions into actions repo
  • Move adapter-agnostic actions into dbt-adapters repo
• Ensure dispatch workflows are idempotent
• Simplify interactions between trigger events and inputs
• Migrate to hatch

Acceptance Criteria

Beta Give feedback

1. Package metadata is defined in pyproject.toml
2. pyproject.toml contains build configuration that supports hatch
3. Relevant scripts or their equivalent are moved into the default hatch environment in pyproject.toml (e.g. from a make file) or are simply retired
4. Test environments use hatch instead of tox
5. pytest configuration is moved into pyproject.toml
6. Requirements are updated, mostly removing old dependencies like tox and bumpversion
7. precommit runs the same formatter and linter as runs as dbt-postgres
8. All workflows adhere to the style standards below
9. This package has the same CI/CD workflows, run options, and concurrency settings as dbt-postgres
10. Branch protection rules are updated, and are limited to a single job, regardless of OS, python version, etc.
11. Integration tests are only configured in one place (e.g. environment variables, etc.)
12. This package is automatically published to dbt's private PyPI daily from main; the current workflow is cancelled in favor of the new workflow when this job is resubmitted
13. This package can be published on demand to the public PyPI, GitHub, and Docker for the specified branch; the current workflow is cancelled in favor of the new workflow when this job is resubmitted for the same branch
14. This package is tested against appropriate published packages prior to releasing
15. I can publish to both test and production versions of publishing targets
16. Publishing to PyPI is authorized via Trusted Publishers
17. Legacy workflows are removed

Suggestions

• Use dbt-postgres as a template
• It's fine to copy paste; it's worth violating DRY for the sake of isolation in a migration of this size

Style standards:

• Job ids, step ids, and variables are in kebab case
• Job names, step names, and description fields follow dbt docs standards (capitalize first word only)
• Extra descriptors should be avoided unless required for disambiguation, e.g.
  • version-number -> version
  • archive-name -> archive
• Workflow files use a four space tab
• Scripts (inline or separate files) use environment variables in env instead of inline substitution like ${{ inputs.value }}

Testing

• New workflows are tested against their dev branch prior to merging

Security

This should not increase security risk, and potentially could reduce it. Any security risks should be addressed as part of dbt-labs/actions#173.

[mikealfare]

This is duplicative of internal tracking.