Redirect to root on consent request #1238

TheAndrewJackson · 2022-10-05T19:27:58Z

Closes #1160

redirect.on.save.mov

Code Changes

Add redirect to index after consents are saved

Steps to Confirm

Confirm user identity
Then save preferences

Pre-Merge Checklist

All CI Pipelines Succeeded (everything but mypy)
Documentation Updated:
- documentation complete, or draft/outline provided (tag docs-team to complete/review on this branch)
- documentation issue created (tag docs-team to complete issue separately)
Issue Requirements are Met
Relevant Follow-Up Issues Created
Update CHANGELOG.md

Description Of Changes

Redirect the user to the index instead of staying on the consent preferences page.

…rn (#873) * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern Refactored the Datastore Management column border layout design to match Figma. * Updated CHANGELOG.md file * 45 - [Datastore Management] Keep datastore cards in 1/3 screen pattern * 745 - [Datastore Management] Keep datastore cards in 1/3 screen pattern * Fixed import statement

* Update config.py * Update crypto imports to fideslib and remove legacy file and tests * Update Identity salt * checkpoint, lowercasing all of the things * update more config values to lowercase * fix linting errors, fix config validators * update the configs to use lowercase keys * lowercase the allowed keys * Update create_test_data.py * bump fideslib version * Lowercase config variables * Fix linting issues * Fix some test failures * Fix application fixtures * Remove old celery config options * Sort script imports * Update changelog Co-authored-by: Thomas <[email protected]>

* Replace config/gh secrets with secrets from vault * sorting and update changelog * fix path for importing test helpers * more sorting * run ci again with empty commit * update unsafe_pr_checks * allow for no vault vars for unit tests, adds back back for saas config toml vars * check for client * sort * init client to None * fix imports * fix pytest markers to better indicate which tests rely on actual secrets

Bumps [pandas](https://github.com/pandas-dev/pandas) from 1.3.3 to 1.4.3. - [Release notes](https://github.com/pandas-dev/pandas/releases) - [Changelog](https://github.com/pandas-dev/pandas/blob/main/RELEASE.md) - [Commits](pandas-dev/pandas@v1.3.3...v1.4.3) --- updated-dependencies: - dependency-name: pandas dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: Paul Sanders <[email protected]>

* initial sendgrid saas connector integration. access only, contacts only * erasure (update) support for sendgrid * remove unused imports * update sendgrid test fixture to expect 404 response status code because of ignore_errors enhancement * Fixing import order and cleaning up the retry logic for consistency * add sendgrid env var support to makefile and unsafe_pr_checks config. remove DELETE endpoint per PR comments * Added delete endpoint for contacts * Fixing data_path for contacts endpoint * Reverting search query to improve performance and avoid server timeouts * Updated delete endpoint request, used request instead of SaaSRequest in tests * updated imports after check suggestion * Updated code after review * Removed unused variables, imports * Restoring Makefile * Fixed import cryptographic_util error * Misc fixes * Updated Changelog file * Updated Changelog for unreleased section and pulled main * Updated Changelog and added Sendgrid in unreleased section * Updated Changelog and added Sendgrid in unreleased section with link * Updated Changelog and added Sendgrid in added section after Adam's suggestion Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Adrian Galvan <[email protected]> Co-authored-by: Hamza W <[email protected]> Co-authored-by: Adrian Galvan <[email protected]>

…ype filter to open up the dropdown (#903) * 747 - Users should be able to click on the full field of a dropdown-type filter to open up the dropdown * Disabled eslint import/extensions rule for certain files with an alias path in the import statement * Updated CHANGELOG.md file * Resolved jest unit tests from failing * Removed eslint-disable import/extensions declarations

Bumps [types-redis](https://github.com/python/typeshed) from 4.3.2 to 4.3.4. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-redis dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* adds saas config base info to connection config responses * update changelog * add properties to base class * use diff naming to avoid recursion * update saas_config prop for unit tests * another occurance of saas config in unit test * use pydantic alias for type field * sort * revert change to saas type naming * unused import * init parent class so that props are avail on self * revert to using props

Bumps [mypy](https://github.com/python/mypy) from 0.961 to 0.971. - [Release notes](https://github.com/python/mypy/releases) - [Commits](python/mypy@v0.961...v0.971) --- updated-dependencies: - dependency-name: mypy dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [pydash](https://github.com/dgilland/pydash) from 5.0.2 to 5.1.0. - [Release notes](https://github.com/dgilland/pydash/releases) - [Changelog](https://github.com/dgilland/pydash/blob/develop/CHANGELOG.rst) - [Commits](dgilland/pydash@v5.0.2...v5.1.0) --- updated-dependencies: - dependency-name: pydash dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](boto/boto3@1.18.14...1.24.34) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [dask](https://github.com/dask/dask) from 2022.6.1 to 2022.7.0. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](dask/dask@2022.6.1...2022.7.0) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…changed for config variables. (#921)

* Correct build arg variable name * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]>

* serve AdminUI by default * updates changelog

* Update python docker base image from slim-buster to slim-bullseye * Update CHANGELOG * Remove ipython from dev-requirements.txt Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]>

Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](boto/boto3@1.24.34...1.24.36) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [faker](https://github.com/joke2k/faker) from 13.15.0 to 13.15.1. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.md) - [Commits](joke2k/faker@v13.15.0...v13.15.1) --- updated-dependencies: - dependency-name: faker dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [types-ujson](https://github.com/python/typeshed) from 5.2.0 to 5.4.0. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-ujson dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add db vs saas to connection type api * adds changelog line * mypy * fix test * format * more tests * formatting * adds system type query param * format * adjust test saas search * correct query param

* Create noxfile.py * update the dockerfile with more stages * add GitPython as a dev requirement (used by nox builds) * add the noxfiles, all docker builds work * fix the "make compose-build" command * remove the worker docker stage due to it being redundant with prod * create a unified compose file for integrations * update the password prompt to be accurate * copy/pasta run_infrastructure into the noxfiles dir so it can be called directly via python * add create_user and seed_test_data to the nox utils * docs commands work * get the generic dev command working * add db commands to utils * clean up mypy configuration in pyproject.toml and remove config from setup.cfg * simplify some of the CI targets and start updating the pytest targets * cleanup run_infrastructure * update ci_suite and other CI nox targets * add nox as a dev-requirement * get dev commands working, sans quickstart * tweak to the compose_down constant * get the new pylint target passing * remove analytics_id * updated the changelog * Apply suggestions from code review Co-authored-by: Paul Sanders <[email protected]> * fix the xenon command * add a few tweaks to the worker dev command * fix an import issue * update dockerignore * move mssql to the bottom of the datastore list Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Paul Sanders <[email protected]>

* Changed Debian base image * Update the python version to be the same as used in the fidesops app image Co-authored-by: Paul Sanders <[email protected]> * Updated changelog Co-authored-by: Dave Quinlan <[email protected]> Co-authored-by: Paul Sanders <[email protected]>

* Change docker password environment variable to MSSQL_SA_PASSWORD * Revert password environment variable name * Set MSSQL user to root in docker-compose * Revert setting root user in docker-compose * Change environment variable name from REQUIRE_MSSQL to SKIP_MSSQL_INSTALLATION * Add healthcheck to mssql compose file * Modify healthcheck command * Revert healthcheck * Try for more loging information * Another try for logging * Another try for logging * Try running only mssql tests to avoid timeout * Revert mssql only flag * Extend time out to try to get logs * Revert extra logging and extended timeout * Set network mode to host * Make mssql run on its own * Remove network from docker-compose and only run mssql in integration * Increase integration test logging * Revert mssql only * Use cache for docker * Fix workflow error * Fix workflow error * Fix target * Fix make traget * Fix make traget * Revert cache * Verify that mssql is running from pytest fixture * Add restart to mssql * Revert rester in docker-compose * Revert wait for mssql in pytest fixture * Wait for mssql to be ready before adding test data * Add count of retries * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]>

* Make nox test name sessions more logical * consolidate the nox test sessions

* Resolve conflicts and upgrade next * Resolve merge conflicts * Fix missing required prop * Update postman collection /policy --> /dsr/policy

* Instead of rebuilding node_modules, just make docker-compose not overwrite them during volume mounting * Increase count for starting up fides, likely needed for M1 macs

…ixtures. (#1153)

Co-authored-by: Adam Sachs <[email protected]>

…d into create_or_update_email_config. (#1146)

…go (#1158) * 1149 - Update all fidesops/fidesctl logos to use the default Fides logo Updated logo.svg file to the new Fides logo * Updated CHANGELOG.md file

* Correctly `isort` `versioneer` in files * Make `optional-requirements.txt` a source of truth * Update `CHANGELOG.md` * Include `optional_requirements.txt` in the package * Specify an encoding when opening requirements files Quiets the pylint warning * Remove superfluous variables * Match dependency names more strictly, add docstring

…res (#1178) * Delete fidesctl user routes in favor of fidesops user routes. They are practically copies of each other except fidesops user logout route has improved logic such as allowing the user to logout with a malformed or expired token. The fidesctl route was taking priority. * Use new CONFIG variable in test_saas_queryconfig * Adjust fidesops health endpoint test. We're using the fidesctl health endpoint now and they surface a version number while fidesops did not. * Add missing CONFIG variable in existing execution tests. Fix incorrect timescale test name. * Add missing timescale secrets to integration_test_config.toml.

…st mode" is True. (#1162)

) * 1177 - ESLint: Parsing error: Cannot read file tsconfig.json file - Resolved parsing error: Cannot read file '/users/ccalhoun/documents/github/fides/tsconfig.json'.eslint - Updated NPM caniuse-lite version dependency - Sorted package.json file * Updated CHANGELOG.md file

* 1191-Duplicate key in fides admin ui package.json file * Updated CHANGELOG.md file

* Stop creating new engines as part of the process of running health checks which cause us to have too many idle connections opened against the application database. Use the same engine that is being shared across the "ops" API endpoints. - Remove unused get_db_for_health_check * Remove the ctl get_db used in the endpoints in favor of the ops get_db. * Mock get_db_health needs two parameters.

* Re-add SaaS template update script to startup hook Also update dependency getter that returns the shared db session directly to be importable from outside classes. * Use a context manager for db session Co-authored-by: Adam Sachs <[email protected]>

* Bump pre-commit from 2.9.3 to 2.20.0 (#853) (#1373) Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 2.9.3 to 2.20.0. - [Release notes](https://github.com/pre-commit/pre-commit/releases) - [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md) - [Commits](pre-commit/pre-commit@v2.9.3...v2.20.0) --- updated-dependencies: - dependency-name: pre-commit dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * 1016 frontend ability for users to manually enter pii to an in progress subject request (#1377) * On successful connector creation the empty yaml text input displays a validation error * Updated caniuse-lite NPM dependency * Added min/max character limitation validation for Manual Webhook DSR Customization * Prevented Chakra Divider component opacity from being overridden at runtime. User could not see visible horizontal line on UI before. * Updated the Manual Processing detail form to not be submitted until the form is dirty. * Removed form dirty check on ManualProcessingDetail component * Update URL when user is navigating to DSR Customization screen when creating a Manual Webhook * Enable retries on saas connectors for failures at the http request level (#1376) * Add decorator to send method that retries throttles * Fix black/mypy * Fix pylint * Add tests for authenticated client * Small fixes and typos * Update CHANGELOG.md * Small changes. Update retry logic to not retry general exceptions Co-authored-by: Eduardo Armendariz <[email protected]> * Add Consent Request API (#1387) Co-authored-by: Paul Sanders <[email protected]> * Add new template for consent requets emails (#1405) * Add new template for consent requets emails * Remove analytics id * Update subject Co-authored-by: Paul Sanders <[email protected]> * Add authenticated route to get consent preferences (#1402) Co-authored-by: Paul Sanders <[email protected]> * Backend: Updating or Deleting Access Webhooks [#1388][#1389] (#1394) * If a manual webhook is deleted or disabled, check if there are any remaining active manual webhooks configured. If not, queue any Privacy Requests stuck in "requires_input" for processing. * In the "view_uploaded_manual_webhook_data", load cached webhook data for a privacy request in strict mode. If it fails (no data saved, extra field saved, field missing), return checked=True, so the user knows they need to reupload data for this webhook before it can be submitted. Return the data in non-strict mode, so we just show the overlap between the data saved and the fields defined. * Update changelog. * Move queue_requires_input_requests to the connection_endpoints where this is the only module it's being called - both where you update and delete a connection config. * Clarify docstring. * Braze Connector: Access Endpoints (#1248) * [#1393] Update Fidesops config with sane defaults where necessary (#1395) * add sane defaults * make subsections of config with complete defaults optional * lowercase database.enabled, set defaults for optional configs * update return type * updates changelog * make PORT an env var * cast env var to int * remove unnecessary unpinned dependency * bump fideslib version * bump fideslib to 3.1.4 * add defaults for the non optional config subclasses * set empty dict to default for config subclasses that require some fields * use .get() in assemble URL for correct error message, correct comment * update jwt_key type annotation * Explain Privacy Request Execution [#1397] (#1396) Co-authored-by: Cole <[email protected]> * Update docker command on privacy center step 4 (#1410) removing the typo ` . at the end of the command so that the command works. * 1319 consent UI api integration (#1407) * Add consent UI back in * Finish initial integration with consent api * WIP consent page * Get initial consent updating working * Improve button look and feel * Add untracked files * Format VerificationForm.tsx * Remove comments * Rename Privacy modal variables * Rename variable * 1401 admin UI persist redux store to localstorage (#1409) * 1401 - Admin UI: Persist Redux store to localStorage * Resolved React memory leak when user attempts to logout via the Subject Requests landing page * Resolved UI unit test failure * Rollback previous change * Removed blacklist property from Redux store config * Refactored due to recommended code review feedback * update footer links (#1406) * update footer links * changelog * Update CHANGELOG.md Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> Co-authored-by: Paul Sanders <[email protected]> * firebase auth integration (#1399) * Initial implementation of firebase auth connector Introduces firebase_admin sdk as a fidesops dependency Leverages request overrides to define custom functionality * Update changelog * Minimize unused client config in firebase auth config * Add user.photo_url as a supported Firebase Auth field Improve readability in access implementation * Fix field name to be more consistent with python SDK properties * Add clarifying comment to unused function docstring * Add photo_url field to firebase auth dataset * Properly test provider data in firebase auth integration. Also include test coverage for delete function, even thought it's not being invoked by default saas config. * Only add fields to result if popluated * Fix empty attribute logic within provider data block * Ignore pylint error for too many variables * fix up provider data check Co-authored-by: Adam Sachs <[email protected]> * Add Braze connector registry entry (#1418) * Braze SVG added * Fixing change log message Co-authored-by: Adrian Galvan <[email protected]> Co-authored-by: Adam Sachs <[email protected]> * Removing PII fields from conversations collection (#1386) * Fidesops v1.8.1 Release Checklist (#1420) * updates changelog * add UI testing steps to release checklist * DX Improvements from `v1.8.1` release (#1421) * stop swallowing integrity errors on dataset update * dont attach logging middleware if analytics opt_out is true * handle diff correctly * disable consider-using-f-string as we need this for logging statements to work effectively in fideslog * Updated CHANGELOG.md file from prior merge * get the server up and running * fix test collection * fix isort and black * update dataset annotations * Merge latest fidesops UI into unified ui * Fixing post UI merge issues * Fix formatting and import issues * get all unit tests passing * Fix cypress failure 🤞 * Remove auth.slice.test.ts Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Eduardo Armendariz <[email protected]> Co-authored-by: Eduardo Armendariz <[email protected]> Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Dawn Pattison <[email protected]> Co-authored-by: Noonari <[email protected]> Co-authored-by: Sean Preston <[email protected]> Co-authored-by: Cole <[email protected]> Co-authored-by: shawnplusplus <[email protected]> Co-authored-by: Andrew Jackson <[email protected]> Co-authored-by: Cole Isaac <[email protected]> Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Adam Sachs <[email protected]> Co-authored-by: Adrian Galvan <[email protected]> Co-authored-by: Thomas <[email protected]>

* Remove SaaS type enum and dynamically reference registered types Update tests to dynamically compare results rather than looking for static values. * Remove references to specific custom SaaSType Since we no longer have an enum for SaaSType, we don't need to have a specific 'custom' type - instead, users can simply create their own type dynamically. There is also now no more invalid 'type' value for saas configs, so we remove the test for that functionality. * Remove unused import * update changelog Co-authored-by: Adam Sachs <[email protected]>

* Bump pydash from 5.0.2 to 5.1.0 (#920) Bumps [pydash](https://github.com/dgilland/pydash) from 5.0.2 to 5.1.0. - [Release notes](https://github.com/dgilland/pydash/releases) - [Changelog](https://github.com/dgilland/pydash/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/dgilland/pydash/compare/v5.0.2...v5.1.0) --- updated-dependencies: - dependency-name: pydash dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update boto3 requirement from ~=1.18.14 to ~=1.24.34 (#917) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.18.14...1.24.34) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump dask from 2022.6.1 to 2022.7.0 (#915) Bumps [dask](https://github.com/dask/dask) from 2022.6.1 to 2022.7.0. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](https://github.com/dask/dask/compare/2022.6.1...2022.7.0) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix integration set up scripts for postgres and mariadb - casing has changed for config variables. (#921) * zendesk and salesforce connection docs (#908) * Adobe Campaign access and erasure (#905) * Updated tutorial to match latest fidesdemo (#772) * Correct build arg variable name (#925) * Correct build arg variable name * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Default `FIDESOPS__ADMIN_UI__ENABLED` to `True` (#936) * serve AdminUI by default * updates changelog * Update python docker base image from slim-buster to slim-bullseye (#928) * Update python docker base image from slim-buster to slim-bullseye * Update CHANGELOG * Remove ipython from dev-requirements.txt Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> * Update boto3 requirement from ~=1.24.34 to ~=1.24.36 (#939) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.24.34...1.24.36) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump faker from 13.15.0 to 13.15.1 (#941) Bumps [faker](https://github.com/joke2k/faker) from 13.15.0 to 13.15.1. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.md) - [Commits](https://github.com/joke2k/faker/compare/v13.15.0...v13.15.1) --- updated-dependencies: - dependency-name: faker dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump types-ujson from 5.2.0 to 5.4.0 (#947) Bumps [types-ujson](https://github.com/python/typeshed) from 5.2.0 to 5.4.0. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-ujson dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add db vs saas to connection type api (#937) * add db vs saas to connection type api * adds changelog line * mypy * fix test * format * more tests * formatting * adds system type query param * format * adjust test saas search * correct query param * Use Nox as the build tool instead of Make (#919) * Create noxfile.py * update the dockerfile with more stages * add GitPython as a dev requirement (used by nox builds) * add the noxfiles, all docker builds work * fix the "make compose-build" command * remove the worker docker stage due to it being redundant with prod * create a unified compose file for integrations * update the password prompt to be accurate * copy/pasta run_infrastructure into the noxfiles dir so it can be called directly via python * add create_user and seed_test_data to the nox utils * docs commands work * get the generic dev command working * add db commands to utils * clean up mypy configuration in pyproject.toml and remove config from setup.cfg * simplify some of the CI targets and start updating the pytest targets * cleanup run_infrastructure * update ci_suite and other CI nox targets * add nox as a dev-requirement * get dev commands working, sans quickstart * tweak to the compose_down constant * get the new pylint target passing * remove analytics_id * updated the changelog * Apply suggestions from code review Co-authored-by: Paul Sanders <[email protected]> * fix the xenon command * add a few tweaks to the worker dev command * fix an import issue * update dockerignore * move mssql to the bottom of the datastore list Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Paul Sanders <[email protected]> * 863 - Retry a DSR (FE) (#938) * Update docs docker base image from slim-buster to slim-bullseye (#949) * Changed Debian base image * Update the python version to be the same as used in the fidesops app image Co-authored-by: Paul Sanders <[email protected]> * Updated changelog Co-authored-by: Dave Quinlan <[email protected]> Co-authored-by: Paul Sanders <[email protected]> * Experimenting with fixes for the failing MSSQL CI tests (#918) * Change docker password environment variable to MSSQL_SA_PASSWORD * Revert password environment variable name * Set MSSQL user to root in docker-compose * Revert setting root user in docker-compose * Change environment variable name from REQUIRE_MSSQL to SKIP_MSSQL_INSTALLATION * Add healthcheck to mssql compose file * Modify healthcheck command * Revert healthcheck * Try for more loging information * Another try for logging * Another try for logging * Try running only mssql tests to avoid timeout * Revert mssql only flag * Extend time out to try to get logs * Revert extra logging and extended timeout * Set network mode to host * Make mssql run on its own * Remove network from docker-compose and only run mssql in integration * Increase integration test logging * Revert mssql only * Use cache for docker * Fix workflow error * Fix workflow error * Fix target * Fix make traget * Fix make traget * Revert cache * Verify that mssql is running from pytest fixture * Add restart to mssql * Revert rester in docker-compose * Revert wait for mssql in pytest fixture * Wait for mssql to be ready before adding test data * Add count of retries * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Delete custom GitHub issue templates (#955) We've defined generic issue templates for all Ethyca repos here: https://github.com/ethyca/.github/tree/main/.github/ISSUE_TEMPLATE Removing the templates from this repo allows it to automatically pull in the organization templates for consistency. * Bump sqlalchemy-redshift from 0.8.8 to 0.8.10 (#940) Bumps [sqlalchemy-redshift](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift) from 0.8.8 to 0.8.10. - [Release notes](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/releases) - [Changelog](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/blob/main/CHANGES.rst) - [Commits](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/compare/0.8.8...0.8.10) --- updated-dependencies: - dependency-name: sqlalchemy-redshift dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Move tests into an "ops" subdir (#935) * Create __init__.py * move all of the test files down one dir into `ops` * update test paths * update paths where needed * update the changelog * Dispatch a repository event on new published releases (#945) * Add release dispatch event action * Update `CHANGELOG.md` * Reduce # of clients connected to the application db [#810] (#944) * Reduce number of open connections: - Limit task concurrency to two per worker. - Create one Engine per celery process which opens up a connection pool. Create one Session per celery process and use that session across privacy requests. - Close the session after the privacy request has finished executing. This just resets the session and returns connections back to the pool. It can be reused. - Remove unnecessary places where session is closed manually because the session is being used as a context manager and is already closed through that. - Pass the same Session that the privacy request is using through to TaskResources to be re-used to create ExecutionLogs instead of opening up a new Session. - Don't close the session when passing it into the Execution Log, wait until the entire privacy request is complete/exited. * Define "self" for run_privacy_task - it's the task itself. For mypy's benefits, define that the session is a context manager. * Make a session non-optional for graph_task.run_access_request, graph_task.run_erasure, and for instantiating taskResources * Use missing db fixture. * Add missing db resource. * Update test to reflect new behavior that disabling a datasource while a request is in progress can cause related collections to be skipped once the current session is expired and the connection config has the most recent state. Because the same Session that is being used to run the PrivacyRequest is now being used for ExecutionLogs, the process of saving an ExecutionLog runs a session.commit() which expires the Session and causes the ConnectionConfig to have the most recent state the next time it is accessed. * Update CHANGELOG. * enable worker by default in our dockerfile (#958) * add extra steps to make clean (#767) * Push `dev` image on pushes to `main` (#956) * Update publish_to_dockerhub.yml * add a dev step and use nox * update the changelog * Move Client Code into an `ops` subdir (#964) * Move Client Code into an `ops` subdir * move all of the files * update the dockerfile * update package.json * update codepaths for workflow tests * Update the changelog and docs references * Update .github/dependabot.yaml * Bump gitpython from 3.1 to 3.1.27 (#971) Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1 to 3.1.27. - [Release notes](https://github.com/gitpython-developers/GitPython/releases) - [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES) - [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.0...3.1.27) --- updated-dependencies: - dependency-name: gitpython dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix the `nox -s dev` command not spinning up the webserver (#959) * Update dev_nox.py * spin up the app before running a container shell * [#927, #929, #930] sendgrid, adobe, outreach connector docs (#951) * Bump sqlalchemy-utils from 0.37.8 to 0.38.3 (#968) Bumps [sqlalchemy-utils](https://github.com/kvesteri/sqlalchemy-utils) from 0.37.8 to 0.38.3. - [Release notes](https://github.com/kvesteri/sqlalchemy-utils/releases) - [Changelog](https://github.com/kvesteri/sqlalchemy-utils/blob/master/CHANGES.rst) - [Commits](https://github.com/kvesteri/sqlalchemy-utils/compare/0.37.8...0.38.3) --- updated-dependencies: - dependency-name: sqlalchemy-utils dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pyodbc from 4.0.32 to 4.0.34 (#980) Bumps [pyodbc](https://github.com/mkleehammer/pyodbc) from 4.0.32 to 4.0.34. - [Release notes](https://github.com/mkleehammer/pyodbc/releases) - [Commits](https://github.com/mkleehammer/pyodbc/compare/4.0.32...4.0.34) --- updated-dependencies: - dependency-name: pyodbc dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump dask from 2022.7.0 to 2022.7.1 (#967) Bumps [dask](https://github.com/dask/dask) from 2022.7.0 to 2022.7.1. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](https://github.com/dask/dask/compare/2022.7.0...2022.7.1) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump alembic from 1.8.0 to 1.8.1 (#989) Bumps [alembic](https://github.com/sqlalchemy/alembic) from 1.8.0 to 1.8.1. - [Release notes](https://github.com/sqlalchemy/alembic/releases) - [Changelog](https://github.com/sqlalchemy/alembic/blob/main/CHANGES) - [Commits](https://github.com/sqlalchemy/alembic/commits) --- updated-dependencies: - dependency-name: alembic dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix erroneous values in the Outreach config (#988) * Add documentation for new nox commands (#981) * Add documentation for new nox commands * changelog * missed make to nox edits * review edits * update `nox -s dev` to not open a shell, add it as a posarg option * Apply suggestions from code review * remove the analytics id * update typo * remove extra numbering Co-authored-by: Thomas <[email protected]> * Bump types-toml from 0.10.7 to 0.10.8 (#998) Bumps [types-toml](https://github.com/python/typeshed) from 0.10.7 to 0.10.8. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-toml dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fideslog from 1.2.1 to 1.2.2 (#996) Bumps [fideslog](https://github.com/ethyca/fideslog) from 1.2.1 to 1.2.2. - [Release notes](https://github.com/ethyca/fideslog/releases) - [Commits](https://github.com/ethyca/fideslog/compare/v1.2.1...v1.2.2) --- updated-dependencies: - dependency-name: fideslog dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Added Choose your connection feature (#987) * Added Choose your connection feature 760-Add a Connection - Select a connector to configure (front end) 866-Add a Connection - Front End layout structure * Updated CHANGELOG.md file * Fix lint issues * Fix build issue * Hide overflow Co-authored-by: Andrew Jackson <[email protected]> * saas request overrides (#986) * initial cut of saas request overrides. include mailchimp as an example and test case. minor refactor of some of the saas request execution to enable smoother override * fix rebase issue by moving saas override tests into ops subdir * import path updates to resolve conflicts caused by rebase * add session parameter into graph task calls to fix saas override integration tests caused by rebase * update changelog * tweaks to saas connector overrides and associated tests * expose override factory register as module variable for clenaer decorator calls Co-authored-by: Adam Sachs <[email protected]> * Update OAuth strategy to be able to perform local testing (#962) * Update boto3 requirement from ~=1.24.36 to ~=1.24.42 (#1001) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.24.36...1.24.42) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fastapi[all] from 0.78.0 to 0.79.0 (#1002) Bumps [fastapi[all]](https://github.com/tiangolo/fastapi) from 0.78.0 to 0.79.0. - [Release notes](https://github.com/tiangolo/fastapi/releases) - [Commits](https://github.com/tiangolo/fastapi/compare/0.78.0...0.79.0) --- updated-dependencies: - dependency-name: fastapi[all] dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update config reference to use lowercase values (#952) * [#909] update config reference to use lowercase values * changelog * missed uppercase variables * update usages of False to false in connector docs * Added Auth0 Connector (#991) * add pagination back to connection types endpoints (#1019) * add pagination back to connection types endpoints * check for pagination in system_type search Co-authored-by: Paul Sanders <[email protected]> * Subject Request Events and Logs Section (#1018) * Fix small issue with eslint config * Get initial drawer working * Add jest config to eslintignore * Fix small css issue * Refactor components and get initial functionality * Fix small logic error * Fix couple of bugs and format code * Conditinally display error tag * Fix issues with merge * Format and lint * Sort imports * Update to new solution * Format code * Update changelog Co-authored-by: Sean Preston <[email protected]> * First draft of OAuth documentation (#963) * Send Errored Requests / Reprocessed Requests Info to FidesLog [#754] (#993) * Add a method to format a representation of the graph for caching in Redis and a separate method to build a summary of the differences in the graph when a privacy request is rerun to fideslog. - Adds FieldAddress.from_string method. * Add methods to cache a representation of the access graph when it is built and a separate method to retrieve it from the cache. - Also add a method to build a "rerun_access_graph" AnalyticsEvent for fideslog where applicable. * When running the access portion of the privacy request, log stats about a rerun and then cache the current access graph. - Give a different prefix when we're caching the access graph to not get mixed up with access request results. * Show skipped new edges that are directly upstream of completed nodes instead. These edges are intentionally dropped from the new graph on rerun, so want to surface this count. * Log if a privacy request fails during the "erasure" step of privacy request execution. Even though the access step is not rerun here, compare the previously cached access graph with the access graph that would have been run to determine what data has changed. * Add missing session variables. * Send an event to Fideslog when privacy request execution fails. * Add missed session - bad merge. * Update changelog. * Update docstrings. * Remove copy/paste comment. * Respond to CR comments. * Currently AnalyticsEvent.local_host cannot be None. * Update the compose file and workflows to expect an already-built image (#966) * Update the compose file and workflows to be in line with fidesctl * update the compose file to look for a specific image * rename files and update the safe PR checks * add check_migrations to the ci checks and nox * fix the failing PR checks * fix CI failures * update the unsafe checks workflow * update run_infrastructure to use the compose service name * remove the makefile and the old run_infra script * bump pylint version, pin isort, fix issues * update pytest setup path * update the changelog * make OPS_TEST_DIR a constant * fix nox missing vars * specify that the nox imports are relative imports * remove relative import paths * run isort Co-authored-by: Sean Preston <[email protected]> * Create `AuditLog` on privacy request approval (#1038) * Create approval audit logs * Add tests * disable a pylint lint * Update changelog * Update privacy_request fixture * Updating Salesforce to use OAuth2 authentication code flow (#1039) * Removing saas_config.toml (#1043) * Bump types-pyyaml from 6.0.9 to 6.0.11 (#1047) Bumps [types-pyyaml](https://github.com/python/typeshed) from 6.0.9 to 6.0.11. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-pyyaml dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump docker/build-push-action from 2 to 3 (#1044) Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 2 to 3. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](https://github.com/docker/build-push-action/compare/v2...v3) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump dask from 2022.7.1 to 2022.8.0 (#1046) Bumps [dask](https://github.com/dask/dask) from 2022.7.1 to 2022.8.0. - [Release notes](https://github.com/dask/dask/releases) - [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md) - [Commits](https://github.com/dask/dask/compare/2022.7.1...2022.8.0) --- updated-dependencies: - dependency-name: dask dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update boto3 requirement from ~=1.24.42 to ~=1.24.46 (#1045) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.24.42...1.24.46) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Refactor static checks to run outside of Docker (#1053) * Refactor static checks to run outside of Docker * run static checks outside of docker, reusing cached virtual envs * update the changelog * Create CHANGELOG.md * Update CHANGELOG.md * update the static checks in the workflows * remove the static check dependency on "build" * Create finished audit log for privacy requests (#1040) * Add finished AuditLog * Bump fideslib * Update test * Fix format lints * Lowercase encoding * Update changelog * Fix issue with migration downgrade * Sort lists in test data tests * format file * Update quickstart to use docker-compose and docker network for all commands (#1056) * Bump snowflake-sqlalchemy from 1.3.2 to 1.3.4 (#1051) Bumps [snowflake-sqlalchemy](https://github.com/snowflakedb/snowflake-sqlalchemy) from 1.3.2 to 1.3.4. - [Release notes](https://github.com/snowflakedb/snowflake-sqlalchemy/releases) - [Commits](https://github.com/snowflakedb/snowflake-sqlalchemy/commits) --- updated-dependencies: - dependency-name: snowflake-sqlalchemy dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * updates changelog (#1063) * Reorganize application code (#1058) * Reorganize application code * first round of moving everything and updating the import paths * checkpoint, more path updates * more path updates * fix imports * fix isort * fix mypy, isort and setup.py issues * fix unsafe checks build step running on any label * update the changelog * delete temp files * Update docs/fidesops/docs/development/contributing_details.md Co-authored-by: Paul Sanders <[email protected]> * update more file references Co-authored-by: Paul Sanders <[email protected]> * Patch versioneer to allow editable installs (#1070) * Patch versioneer to allow editable installs * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Preserving headers in SaaSRequestParams during pagination (#1069) * Add setuptools to dev-requirements to fix versioneer error (#1072) * Add setuptools to dev-requirements to fix versioneer error and revert patch * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Bump fideslang to 1.1.0 (#890) * Bump fideslang to 1.1.0 * find/replace user.provided.identifiable -> user * remove derived mentions * don't remove volumes on teardown * update address fields * replace user.derived and user.provided -> user * fix two more tests * fix pylint errors * fix integration tests * fix failing mongo tasks * update the changelog * fix the failing mongo task test * another mongo task fix * more mongo task fixes * Revert test back to two addresses being masked. * Update mongo array access test to reflect that underlying dataset has changed, and policy has changed, so more fields are returned. * add the noxfiles * update the dockerfile and get the nox docker commands working * Revert "update the dockerfile and get the nox docker commands working" This reverts commit 4b98c62163d419996977b7bb2dd17d181aac2f07. * remove noxfiles * updates from comments * Update test * Add migration * Update categories in test config files * Fix data categories * Fix more data categories * Change user.provided.nonidentifiable to user * Update migraiton with review suggestions * Run black * Add more logging to migration * Increment counter * fix migration conflict Co-authored-by: Dawn Pattison <[email protected]> Co-authored-by: Paul Sanders <[email protected]> * 797 datadog (#1060) * 515 SaaS connector Logi ID (#1074) * Combine Execution and Audit Logs in Request Status Endpoint [#1024] (#1068) * Update the request status endpoint, so when the verbose query param is used and execution logs are embedded, also return audit logs. Execution Logs are created at the collection level while audit logs are for the overall privacy request level, so most fields returned for audit logs are None. Logs are also grouped at the dataset level here, so give the audit logs a fake dataset name for display purposes, for example, "Request approved". * Update CHANGELOG and update docs to reflect that audit logs are included in a verbose request status response. * Notify fidesdemo on new releases (#1075) * Notify fidesdemo on new releases * Update `CHANGELOG.md` Co-authored-by: Paul Sanders <[email protected]> * Bump pytest from 6.2.2 to 7.1.2 (#1081) Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.2.2 to 7.1.2. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pytest-dev/pytest/compare/6.2.2...7.1.2) --- updated-dependencies: - dependency-name: pytest dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump faker from 13.15.1 to 14.0.0 (#1080) Bumps [faker](https://github.com/joke2k/faker) from 13.15.1 to 14.0.0. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.md) - [Commits](https://github.com/joke2k/faker/compare/v13.15.1...v14.0.0) --- updated-dependencies: - dependency-name: faker dependency-type: direct:development update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 3 to 4 (#1078) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3 to 4. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](https://github.com/actions/setup-python/compare/v3...v4) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fideslog from 1.2.2 to 1.2.3 (#1079) Bumps [fideslog](https://github.com/ethyca/fideslog) from 1.2.2 to 1.2.3. - [Release notes](https://github.com/ethyca/fideslog/releases) - [Commits](https://github.com/ethyca/fideslog/compare/v1.2.2...v1.2.3) --- updated-dependencies: - dependency-name: fideslog dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump types-urllib3 from 1.26.15 to 1.26.22 (#1084) Bumps [types-urllib3](https://github.com/python/typeshed) from 1.26.15 to 1.26.22. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-urllib3 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fideslang from 1.1.0 to 1.2.0 (#1085) Bumps [fideslang](https://github.com/ethyca/fideslang) from 1.1.0 to 1.2.0. - [Release notes](https://github.com/ethyca/fideslang/releases) - [Changelog](https://github.com/ethyca/fideslang/blob/main/CHANGELOG.md) - [Commits](https://github.com/ethyca/fideslang/compare/1.1.0...1.2.0) --- updated-dependencies: - dependency-name: fideslang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump next-auth from 4.9.0 to 4.10.3 in /clients/ops/privacy-center (#1014) Bumps [next-auth](https://github.com/nextauthjs/next-auth) from 4.9.0 to 4.10.3. - [Release notes](https://github.com/nextauthjs/next-auth/releases) - [Changelog](https://github.com/nextauthjs/next-auth/blob/main/CHANGELOG.md) - [Commits](https://github.com/nextauthjs/next-auth/compare/[email protected]@v4.10.3) --- updated-dependencies: - dependency-name: next-auth dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sean Preston <[email protected]> * Bump fideslib (#1092) * Bump fideslib to fix docs auth issue * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * 983 Adds infra for email config and dispatch (#1059) * Email config mvp crud / db layer * adds name to email config model * remove unintended changes * gets POC working * removes org name config var, updates crud endpoints to better handle supporting only 1 config, formatting * updates postman collection, adds to changelog, bumps downrev on migration * sort * formatting * use correct response model * Delete base.py * update migration annotation * Create a Saas Connector from a Template [#814] (#1076) * Starting point for SaaS connector templates * Fix imports from restructuring. * Get happy path working for instantiate connector from template endpoint. * Remove updating connector instances for now - out of scope. * Test nonexistent templates, secrets validation, instance key / fides key already exists. * Create DatasetConfigs and ConnectionConfigs instead of create_or_update in the template endpoint. Don't save ConnectionConfig until secrets are validated. * Add the other saas connectors to the registry and update their configs and datasets with instance_fides_key. - Fix datadog yaml so it can be included in the saas connector registry. There was an error in how the saas config was formatted. * Update the fides_keys in the existing saas configs and dataset yamls to have brackets around the "instance_fides_key" to indicate these will be replaced. Update the fides_key definition to allow "<instance_fides_key>" with brackets specifically to pass validation. * Fix a side effect on a separate endpoint that returns the types of secrets that should be supplied for a given connector. Use the saas config type instead of the fides key for the model title. Add test verifying that fides key /instance key validation works as expected. * - Update CHANGELOG - Add new endpoint to postman collection - Add drafts doc. - Update old response body in docs for connection types. * Replace the <instance_fides_key> with a properly formatted fides_key in the saas fixtures. * If DatasetConfig creation fails, delete the recently created ConnectionConfig. * Address some of the saas integration tests where I've changed the fides_key. * Fix typos. * Fix typo. * Fix unrelated bug where hubspot dataset has new datacategories with user-* data categories after the fideslang update, so they would show up if the user picked a "user" data category. * Respond to CR. Co-authored-by: Dawn Pattison <[email protected]> * Update the "instantiate_connection_from_template" to return a portion of both the connection config and the dataset. (#1105) * Adds `AuditLog` and `ExecutionLog`s to `seed_test_data` command (#1097) * Reduce Idle Connections from Health Checks [#1102] (#1107) * Don't create a new engine as part of running the health checks and share a single engine across the application, including for the health checks. Currently we're using the default pool_size and max_overflow. * Update changelog. * Fix that health checks are still supposed to run, even if the database is disabled. * Need to yield instead - 'generator' object has no attribute 'query' * Escape redis user and password (#1104) * Escape redis user and password * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * pass in analytics id env to worker (#1113) * pass in analytics id env to worker * changelog * Adds tests for email endpoints and dispatch service (#1112) * Endpoint to Verify User Identity [#1062] (#1111) * Add an endpoint to verify a user's identity before queuing the privacy request provided it doesn't need separate manual approval by a system admin. - Add a new PrivacyRequest.identity_verified_at timestamp - Add a new PrivacyRequestStatus - "identity_unverified". - Add methods to cache the verification code in Redis for comparison with a default ttl of 10 minutes * - Fix linting/copy-paste errors. - Update changelog. - Add endpoint to postman * Add new keys to response bodies. * Instead of using a new VerificationCode schema, use the SubjectIdentityVerificationBodyParams that already exists. * Revert "Instead of using a new VerificationCode schema, use the SubjectIdentityVerificationBodyParams that already exists." This reverts commit 40fcf6d119135d08a6d3ecfc40c5d73846bf2205. * Bump hashicorp/vault-action from 2.4.1 to 2.4.2 (#1119) Bumps [hashicorp/vault-action](https://github.com/hashicorp/vault-action) from 2.4.1 to 2.4.2. - [Release notes](https://github.com/hashicorp/vault-action/releases) - [Changelog](https://github.com/hashicorp/vault-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/hashicorp/vault-action/compare/v2.4.1...v2.4.2) --- updated-dependencies: - dependency-name: hashicorp/vault-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fastapi[all] from 0.79.0 to 0.79.1 (#1120) Bumps [fastapi[all]](https://github.com/tiangolo/fastapi) from 0.79.0 to 0.79.1. - [Release notes](https://github.com/tiangolo/fastapi/releases) - [Commits](https://github.com/tiangolo/fastapi/compare/0.79.0...0.79.1) --- updated-dependencies: - dependency-name: fastapi[all] dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump types-urllib3 from 1.26.22 to 1.26.23 (#1117) Bumps [types-urllib3](https://github.com/python/typeshed) from 1.26.22 to 1.26.23. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-urllib3 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump nox from 2022.1.7 to 2022.8.7 (#1118) Bumps [nox](https://github.com/wntrblm/nox) from 2022.1.7 to 2022.8.7. - [Release notes](https://github.com/wntrblm/nox/releases) - [Changelog](https://github.com/wntrblm/nox/blob/main/CHANGELOG.md) - [Commits](https://github.com/wntrblm/nox/compare/2022.1.7...2022.8.7) --- updated-dependencies: - dependency-name: nox dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump faker from 14.0.0 to 14.1.0 (#1122) Bumps [faker](https://github.com/joke2k/faker) from 14.0.0 to 14.1.0. - [Release notes](https://github.com/joke2k/faker/releases) - [Changelog](https://github.com/joke2k/faker/blob/master/CHANGELOG.md) - [Commits](https://github.com/joke2k/faker/compare/v14.0.0...v14.1.0) --- updated-dependencies: - dependency-name: faker dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump sqlalchemy-redshift from 0.8.10 to 0.8.11 (#1121) Bumps [sqlalchemy-redshift](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift) from 0.8.10 to 0.8.11. - [Release notes](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/releases) - [Changelog](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/blob/main/CHANGES.rst) - [Commits](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift/compare/0.8.10...0.8.11) --- updated-dependencies: - dependency-name: sqlalchemy-redshift dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * POC for dynamic routing (#1033) * Add initial POC for dynamic routing * fix a couple lints * Fix mypy lint * making pylint happy * Remove log * Fix another pylint issue * Add docstring * Update index if check * Handle nested nextjs routes * Update changelog * Add Sass connector configuration (#1099) * Sass Connecter feature development 761 - Add a Connection - SaaS connector's configuration parameters 984 - Saas Connector configuration - left navigation to toggle between connection params and dataset config 985 - SaaS Connector Configuration - Testing a Connection * Skipping unit test temporarily * Update flags.json file By default, turning off the createNewConnection flag which is still under development. * Updated CHANGELOG.md file * Resolved ESLint issues * Resolved npm build issue * Resolved npm build issue * Updated Saas connector configuration * Updated Saas connector configuration * Removed unnecessary import statement * add new privilege for creating SaaS connectors to user management interface * add connections read as a privilege * Applied code review feedback * Updated the CSS visibility of the CircleHelpIcon component * Added toast success when a user creates a Saas config * Resolved ESLint issue Co-authored-by: Sean Preston <[email protected]> * 1128-Add Retry button back into the subject request detail view (#1131) * 1128-Add Retry button back into the subject request detail view * Updated CHANGELOG.md file * provide a way to give invited users the resume permission Co-authored-by: Sean Preston <[email protected]> * Fix localhost setting for non-endpoint analytics calls [#1029] (#1130) * Set local_host to None for non-endpoint analytics calls. These are logging various tasks coming out of celery. * Update Changelog. * Add email_templates module (#1123) * Add email_templates module * run isort * Add unit tests * Update ttl calculation * Add ttl minutes test * fix lint issues * fix pylint issue * fix pylint issue * fix isort * Update template constant * Update changelog * fix lints * Add jinja to requirements.txt * update templates directory * update unit test * Update imports * fix issue template path * Add templates to manifest Co-authored-by: Paul Sanders <[email protected]> * Formatting Issues in Webhooks Documentation (#1114) * Fix typo in `derived_identity` * Update CHANGELOG.md * Missed PR Link * replaced `<>` with `{}` In the live version of the docs, the `<>`s were being stripped from our code example titles. I've replaced them with `{}` to align with some of the other pages I've seen. * Send Identity Verification Email [#1010] (#1115) * If identity verification required, send email to the user with the verification code. * Adjust the identity_verification_required autouse fixture, and add an autouse override for just the tests where we want to turn on identity verification. * Add starting docs and updating the changelog. Start with identity_verification_required set to False for now until all the related pieces are in. * Update some of the docstrings. * Add unverified status color in the FE. * Add new privacy request status to types and constants. * Restore trailing comma. * Update identity_verification_required to subject_identity_verification_required for clarity. * Adjust email_body_params to accommodate new template. Co-authored-by: Sean Preston <[email protected]> * Support case-insensitive connection type search [Unticketed] (#1133) * Make connection type search case-insensitive. * Update changelog. * Add option to login as root user from config (#1116) Co-authored-by: Paul Sanders <[email protected]> * Move logging to %-style formatting [#837] (#1132) * Add a new Pii class and use it to wrap arguments not already wrapped with NonPii in those logs that are currently using %-style formatting. * Switch logging formatting to %-style instead of f-string. * Continue to address lingering f string instances, and wrap some arguments in Pii, such as raw exceptions. * Remove NotPii class and update tests. * Adjust errors made in %-style conversion. * Remove accidental Pii on print statements, update some PII wrappings. * Adjust string formatting of newly added log. * Update Changelog. * Fix missed closing curly brackets. * Remove missed curly brackets. * Add future annotations for backwards compatibility (#1136) * Add future annotations for backwards compatibility * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Release 1.7.1 (#1141) * update changelog * update ERD * make Hubspot test more robust * Fix docs build in CI (#1138) * Fix docs build in CI * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> Co-authored-by: Sean Preston <[email protected]> * fix command syntax (#1143) * hook up API request to logout action [Unticketed] (#1139) * hook up API request to logout action * run format * update import orer * add lint:fix * Update hubspot users and owners configurations (#1091) * Update CHANGELOG.md * SaaS Connector Template Creation Fix: Integer fides_key (#1166) * Use quotes when replacing <"instance_fides_key"> in the saas config and dataset config files to force a string. * Update CHANGELOG. * Start a new "email" ConnectionConfig type [#1134] (#1142) * Start a new "email" ConnectionConfig type. * Hide "email" type from the get_connection_types endpoint for now, as the email connector isn't fleshed out yet. * Update CHANGELOG. * Simplify by sending one email to start? * Update request body in postman collection. * Fix CHANGELOG formatting. * Revert "SaaS Connector Template Creation Fix: Integer fides_key (#1166)" (#1171) This reverts commit 28b6317fc7bf86019eb6559824226dc261511a95. * Fix analytics opt out environment variable name (#1170) * 1004 OAuth2 client credentials flow (#1159) * update Subject Request detail page (#1164) * [#495] clarify additions to subject request event log * [#863] note retry capabilities in subject requests * changelog * fidesops.toml * changelog Co-authored-by: Cole Garbo <[email protected]> * Bump fastapi[all] from 0.79.1 to 0.81.0 (#1178) Bumps [fastapi[all]](https://github.com/tiangolo/fastapi) from 0.79.1 to 0.81.0. - [Release notes](https://github.com/tiangolo/fastapi/releases) - [Commits](https://github.com/tiangolo/fastapi/compare/0.79.1...0.81.0) --- updated-dependencies: - dependency-name: fastapi[all] dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Build docker image for privacy center (#1165) * Build docker image for privacy center * Update docs and add docker publish to workflow * Update CHANGELOG * Move privacy center docker publish to its own workflow Co-authored-by: Paul Sanders <[email protected]> * Update `StorageConfig` to handle multiple auth methods (#1140) * Initial automatic impl * Lints * disable pylint error * Fix unit tests * fix imports * Update fixture * Fix config tests * Covert type to enum * Add new unit tests * Update changelong * Update docs * Update phone number * patch task scheduler * remove pylint ignore and add exception * Update docs * 1009 id verification required endpoint (#1221) * Adds identity verification config endpoint * adds test, reuse existing config var, adds postman collection * update changelog * add type ignore * move ignore line * Update src/fidesops/ops/graph/config.py Co-authored-by: Paul Sanders <[email protected]> * remove duplicate lines in method * another dupe line * remove scope, update order of classmethod decorator * remove dependencies on endpoint * unused imports Co-authored-by: Paul Sanders <[email protected]> * fix reference in data categories upgrade migration (#1223) * Publish on every release (#1226) Co-authored-by: Paul Sanders <[email protected]> * Make log send async (#1174) * Make log send async * Update CHANGELOG * Add async to additional fideslog calls * WIP * Fix issue with async function is celery * Make __send work with name mangling * Remove extra await * Await coroutines in tests * Remove analytics id Co-authored-by: Paul Sanders <[email protected]> * 1106 saas config shopify access endpoints (#1220) * Adds email communications guide to docs, exposes other hidden guides in nav bar (#1233) * adds email communications guide to docs, exposes other hidden guides in nav bar * update changelog * email docs copyedits * remove dupe items from nav Co-authored-by: Cole <[email protected]> * Fix version number (#1232) Co-authored-by: Paul Sanders <[email protected]> * Bump pylint from 2.14.5 to 2.15.0 (#1182) * Bump pylint from 2.14.5 to 2.15.0 Bumps [pylint](https://github.com/PyCQA/pylint) from 2.14.5 to 2.15.0. - [Release notes](https://github.com/PyCQA/pylint/releases) - [Commits](https://github.com/PyCQA/pylint/compare/v2.14.5...v2.15.0) --- updated-dependencies: - dependency-name: pylint dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> * Ignore missing-timeout warning Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Paul Sanders <[email protected]> * Bump fideslib (#1236) * Bump fideslib * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]> * Email Connector: Build Masking Instructions (#1168) * Pass in input_data to erasure requests, and not just access requests, so it can be used for the email connector, which won't have any rows returned from an access request. - Add an EmailConnector.build_masking_instructions method with a draft of data needed to instruct the user how to query/mask/what fields to mask on their end. * Have the EmailConnector.mask_data cache the raw details of what needs to be masked in Redis. We'll use this to send one email at the end for each "email"-based dataset at end, instead of sending one email for each collection. Reuse some of the caching code created for manual connectors / failed privacy requests where similar to the EmailConnectors, we have some separate action that is required on a given collection. Rename to make more generic. * Remove restriction that a ManualAction needs a get or update value. The manual action could just be locating data for another collection downstream. Cache email template details, even if there are no actions needed on that specific collection, * Update the expected number of collections in the email dataset. * build_masking_instructions is not required to return a ManualAction. * Reconcile this test with the work to make log send asynchronous. Co-authored-by: Sean Preston <[email protected]> * Adds email scopes to postman collection (#1241) * Adds email scopes to postman collection * update changelog * Bump black from 22.6.0 to 22.8.0 (#1238) Bumps [black](https://github.com/psf/black) from 22.6.0 to 22.8.0. - [Release notes](https://github.com/psf/black/releases) - [Changelog](https://github.com/psf/black/blob/main/CHANGES.md) - [Commits](https://github.com/psf/black/compare/22.6.0...22.8.0) --- updated-dependencies: - dependency-name: black dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Rename fidesops-privacy-center image to fides-privacy-center (#1237) Co-authored-by: Paul Sanders <[email protected]> * Bump eslint from 8.9.0 to 8.23.0 in /clients/ops/privacy-center (#1180) Bumps [eslint](https://github.com/eslint/eslint) from 8.9.0 to 8.23.0. - [Release notes](https://github.com/eslint/eslint/releases) - [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md) - [Commits](https://github.com/eslint/eslint/compare/v8.9.0...v8.23.0) --- updated-dependencies: - dependency-name: eslint dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * #1008 User identity verification flow (#1231) * Update lint commands and eslint ignore * Run lints and refactor privacy cards * Refactor modal * Finish verification flow * get config from server * Update changelong * Fix test failures * Format file * Mock out route * Format file * Add code resending * Update test to use hostUrl * Add headers util function and PrivacyRequestStatus status enum * Bump @chakra-ui/utils from 1.10.4 to 2.0.9 in /clients/ops/admin-ui (#1145) Bumps [@chakra-ui/utils](https://github.com/chakra-ui/chakra-ui/tree/HEAD/packages/utils) from 1.10.4 to 2.0.9. - [Release notes](https://github.com/chakra-ui/chakra-ui/releases) - [Changelog](https://github.com/chakra-ui/chakra-ui/blob/main/packages/utils/CHANGELOG.md) - [Commits](https://github.com/chakra-ui/chakra-ui/commits/@chakra-ui/[email protected]/packages/utils) --- updated-dependencies: - dependency-name: "@chakra-ui/utils" dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Allows worker to start successfully in both dev and dev_with_worker (#1250) * Allows worker to start successfully in both dev and dev_with_worker * update changelog * formatting * Add `execution_timeframe` to `Policy` model and schema (#1244) * Add execution_timeframe column to model and schema * Fix test failures * Fix Policy patch route * Update Policy patch test * Run black and isort lints * Update changelog * fix typo Co-authored-by: Sean Preston <[email protected]> * Fix for pytest-asyncio bug (#1260) Co-authored-by: Paul Sanders <[email protected]> * Clean up docker build (#1252) * Add files to .dockerignore * Add config path to docker-compose * Mount volume for build check * Fix pylint error * Update CHANGELOG * Fix failing external test * Remove worker env var from docker-compose Co-authored-by: Paul Sanders <[email protected]> * Email Connector: Send Email with Erasure Instructions [#1158] (#1246) * Send an email for each email-based dataset at the end of privacy request execution. - Add a migration to create a new audit log type. Create an audit log for the email send. - Throw an exception for email-based connectors and catch to override the default execution log. - Add a draft of an email template - Connect sending a "test email" with dummy data. A fidesops admin could configure to check their email config was working. * Add more "checkpoints" to privacy request execution - these are locations from which we can resume privacy request execution without having to run from the beginning. - Add more options to CurrentStep Enum - Cache the checkpoint if an email send fails, so we can retry from the same step. * Don't send an email if the connection config is read only or there are no updates to be applied to any of the collections on the dataset. * Don't assume there's a collection when building "resume" details. A failed privacy request can be resumed outside of the traversal. * Add a first draft of docs for setting up an email connector. * Moves the email connector send method to the email connector file. * Update mock location. * Bump downrev. * update email connector guides * correct link, broken sentence * Create a new EmailRequestFulfillmentBodyParams type to be used once the cached email details are extracted by dataset. * Fix missed test. Co-authored-by: Cole <[email protected]> * Fix download link (#1264) * Fix download link * Update changelog * Add `due_date` to `PrivacyRequest` model (#1259) * Add `due_date` to `PrivacyRequest` model * Add `due_date` test and run lints * Update CHANGELOG.md * Fix test failures * Fix type error * Fix mypy issue * Switch to `strptime` * Fix migration down revision * Move date format string into constant * Refactor strategy instantiation for more extensitiliby (#1254) * Instantiate strategies via abstract Strategy base class A generalized Strategy abstract base class provides generalized getter methods that instantiate strategy subclasses (implementations). These methods rely on the builtin __subclasses__() method to identify Strategy subclasses, which allows for more dynamic and extensible strategy implementation, removing the need for a hardcoded enumeration of supported Strategy implementations. Abstract strategy types inherit from this new abstract base class, and strategy subclasses (implementations) must provide `name` and `configuration_model` attributes that are leveraged by new instantiation mechanism in the abstract base class. * Update get_description() to be a class rather than static method This allows the method to leverage the new `name` class variable rather than relying on a static constant variable. * Remove strategy factories and update references Strategy factories are no longer needed with refactored Strategy getters. Update the uses (references) of strategy factories throughout the codebase to now rely on the new Strategy getters. Strategy subclasses (implementations) now need to be imported explicitly in __init__.py's because they used to be imported in factory modules. Also remove the old MaskingStrategy registration/factory mechanisms. * Remove strategy name constants Now that the abstract Strategy base class enforces implementation subclasses to have a `name` class attribute, this attribute should be relied upon rather than the arbitrary name constants declared previously. The get_strategy_name() abstract method is also superfluous, as the `name` class attribute can be used as a standardized way to retrieve the strategy name. * Remove get_configuration_model() abstract method The generalized strategy getter now relies upon the `configuration_model` class variable that's on each Strategy. Therefore we no longer need the get_configuration_model() getter on each Strategy subclass. * Update MaskingStrategy docs with new Strategy functionality * Update changelog * Improve recursion in _find_all_strategy_subclasses * Fix recursion bug when finding all strategies Update associated tests to make sure the recursion is properly tested * Tweak conditional for falsy check * Make get_strategies endpoint test more robust * Fix typo in documentation Co-authored-by: Adam Sachs <[email protected]> * Make admin ui work when volumes are mounted (#1266) * Make admin ui work when volumes are mounted * Update changelog * Fix path issue with hard refresh * Update from code review Co-authored-by: Paul Sanders <[email protected]> * Update the Erasure Request Email Fulfillment template [#1265] (#1270) * Update email template for email request fulfilment. * Formatting changes. * Bump types-redis from 4.3.4 to 4.3.20 (#1255) Bumps [types-redis](https://github.com/python/typeshed) from 4.3.4 to 4.3.20. - [Release notes](https://github.com/python/typeshed/releases) - [Commits](https://github.com/python/typeshed/commits) --- updated-dependencies: - dependency-name: types-redis dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update boto3 requirement from ~=1.24.46 to ~=1.24.68 (#1272) Updates the requirements on [boto3](https://github.com/boto/boto3) to permit the latest version. - [Release notes](https://github.com/boto/boto3/releases) - [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst) - [Commits](https://github.com/boto/boto3/compare/1.24.46...1.24.68) --- updated-dependencies: - dependency-name: boto3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump requests-mock from 1.9.3 to 1.10.0 (#1276) Bumps [requests-mock](https://github.com/jamielennox/requests-mock) from 1.9.3 to 1.10.0. - [Release notes](https://github.com/jamielennox/requests-mock/releases) - [Commits](https://github.com/jamielennox/requests-mock/compare/1.9.3...1.10.0) --- updated-dependencies: - dependency-name: requests-mock dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fastapi[all] from 0.81.0 to 0.82.0 (#1277) Bumps [fastapi[all]](https://github.com/tiangolo/fastapi) from 0.81.0 to 0.82.0. - [Release notes](https://github.com/tiangolo/fastapi/releases) - [Commits](https://github.com/tiangolo/fastapi/compare/0.81.0...0.82.0) --- updated-dependencies: - dependency-name: fastapi[all] dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot…

chriscalhoun1974 and others added 30 commits July 18, 2022 09:06

Add dependabot label to dependabot PRs (#898)

63ea38d

Co-authored-by: Paul Sanders <[email protected]>

[SaaS Connector] Salesforce (erasure) (#888)

61b258e

Adding privacy_request_id placeholder (#911)

8068597

Fix integration set up scripts for postgres and mariadb - casing has …

100b939

…changed for config variables. (#921)

zendesk and salesforce connection docs (#908)

8cc0fef

Adobe Campaign access and erasure (#905)

a49bd88

Updated tutorial to match latest fidesdemo (#772)

1c1db65

Correct build arg variable name (#925)

175cf81

* Correct build arg variable name * Update CHANGELOG Co-authored-by: Paul Sanders <[email protected]>

Default FIDESOPS__ADMIN_UI__ENABLED to True (#936)

343d36e

* serve AdminUI by default * updates changelog

Add db vs saas to connection type api (#937)

744d7e8

* add db vs saas to connection type api * adds changelog line * mypy * fix test * format * more tests * formatting * adds system type query param * format * adjust test saas search * correct query param

863 - Retry a DSR (FE) (#938)

0ad5a17

ThomasLaPiana and others added 22 commits September 29, 2022 09:28

fix docs merge conflicts

51decf0

remove the analytics_id

9181b2a

Make nox test name sessions more logical (#1135)

bec0fdd

* Make nox test name sessions more logical * consolidate the nox test sessions

Fix privacy center conflicts (#1133)

1ddeb54

* Resolve conflicts and upgrade next * Resolve merge conflicts * Fix missing required prop * Update postman collection /policy --> /dsr/policy

Make local docker UI better (#1140)

8093ee5

* Instead of rebuilding node_modules, just make docker-compose not overwrite them during volume mounting * Increase count for starting up fides, likely needed for M1 macs

Disable various email sends by default in unit testing with autouse f…

6d008cb

…ixtures. (#1153)

add vault environment variables to docker-compose container (#1151)

884517f

Co-authored-by: Adam Sachs <[email protected]>

FIdesConfig instead of the EmailConfigRequest resource is being passe…

8e460ea

…d into create_or_update_email_config. (#1146)

1149 - Update all fidesops/fidesctl logos to use the default Fides lo…

9e9b421

…go (#1158) * 1149 - Update all fidesops/fidesctl logos to use the default Fides logo Updated logo.svg file to the new Fides logo * Updated CHANGELOG.md file

ui/package: restore copy-export command (#1166)

990d4a9

Only show parameters inside queries if "dev mode" is True, not if "te…

167ea70

…st mode" is True. (#1162)

1191 duplicate key in fides admin UI package.json file (#1192)

a22b541

* 1191-Duplicate key in fides admin ui package.json file * Updated CHANGELOG.md file

Square access and erasure (#1193)

cd62aec

Redirect to root on consent request

35be57b

Update CHANGELOG.md

e3f5fc4

TheAndrewJackson marked this pull request as ready for review October 5, 2022 20:46

Base automatically changed from unified-fides-2 to main October 6, 2022 16:18

ThomasLaPiana requested review from a team October 6, 2022 16:18

TheAndrewJackson mentioned this pull request Oct 6, 2022

Move /clients/ops/privacy-center to /clients/privacy-center #1260

Closed

TheAndrewJackson closed this Oct 6, 2022

TheAndrewJackson deleted the 1160_privacy_center_redirect branch October 11, 2022 10:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Redirect to root on consent request #1238

Redirect to root on consent request #1238

TheAndrewJackson commented Oct 5, 2022 •

edited

Loading

Redirect to root on consent request #1238

Redirect to root on consent request #1238

Conversation

TheAndrewJackson commented Oct 5, 2022 • edited Loading

Code Changes

Steps to Confirm

Pre-Merge Checklist

Description Of Changes

TheAndrewJackson commented Oct 5, 2022 •

edited

Loading