Create initial set of "test setup scripts" to configure local Fides environment with basic data

[NevilleS]

Description

Create a default set of “test setup scripts” that configure a Fides instance with enough data to exercise all core features, including:

• custom Fides resources: taxonomies, systems, datasets, policies
• Admin UI users
• DSR connectors: database, SaaS, manual
• DSR policies (privacy request rules)
• privacy requests (in all statuses)
• storage & email configs
• etc.

Note that this will also need a clean, standard way to inject secrets for connectors, user passwords, 3rd party test systems, etc.

We've built several versions of these kinds of scripts, but have never invested enough time & energy to making them maintainable. One of the main reasons is that we've not built automated tests that rely on the scripts enough to force them to keep working and stay up to date. For some examples, see:

• the fidesdemo repo: https://github.com/ethyca/fidesdemo/blob/main/flaskr/fidesops.py
• the fidesops quickstart: https://github.com/ethyca/fides/blob/main/scripts/quickstart.py
• the seed_test_data command: https://github.com/ethyca/fides/blob/main/scripts/create_test_data.py
• etc.

These scripts will be used for at least three purposes over time:

1. Setting up a local environment with test data for engineers & product managers to test features
2. Setting up a staging environment with test data for automated CI checks to test features
3. Setting up a hosted environment with test data for giving product & sales demos

Because of this, it's important to make these scripts as portable as possible; ideally, these are relatively vanilla Python scripts that just leverage Fides CLI & API commands without calling source code functions at all.

Acceptance Criteria

Test Setup Scripts Framework

• MUST save setup scripts in a directory structure that is easy to intuit and maintain, e.g. scripts/setup/systems.py, scripts/setup/postgres_connector.py, or similar
• MUST have a nox -s setup_test (or similar name!) that runs all "basic" scripts to fully configure the local Fides database
• MUST NOT rely on pytest features or functionality; these scripts are intended to be reused in the future for E2E runners (e.g. Cypress) and deployed environments without pytest
• MUST inject secrets using ENV variables for reusability (NOTE: within nox, these ENV vars could be auto-populated by Vault)

Basic Test Setup Scripts

To get our local environment populated with enough "basic" data to test our core features, we MUST have scripts that:

1. configure a "Fides API client" with OAuth token to test protected APIs
2. configure Users to test the Admin UI
3. configure Systems to test the data map
4. configure Datasets to test the data map
5. configure custom Data Categories, Data Uses, and Data Subjects to test the data map
6. configure an S3 Storage location to test privacy requests
7. configure a Mailgun email to test privacy requests
8. configure identity verification for the privacy center to test privacy requests
9. configure DSR access & erasure policies to test privacy requests
10. configure a Postgres connector (and dataset) to test privacy requests (using existing integration test scripts for Postgres)
11. configure a Mongodb connector (and dataset) to test privacy requests (using existing integration test scripts for Mongodb)
12. configure a Mailchimp connector (and dataset) to test privacy requests (using existing integration test scripts for Mailchimp)
13. configure a Stripe connector (and dataset) to test privacy requests (using existing integration test scripts for Stripe)
14. configure privacy requests in various statuses

Manual Testing

Once all basic setup scripts have run, a tester should be able to:

1. Login to the Admin UI as a User and view existing Users in User Management UI
2. View existing Systems, Datasets, and custom Taxonomy resources in their respective screens
3. View existing Privacy Requests and Connectors in their respective screens
4. Submit and execute a new privacy request via the Privacy Center & Admin UI