Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates for consent signal processing #5200

Merged
merged 37 commits into from
Aug 27, 2024

Conversation

galvana
Copy link
Contributor

@galvana galvana commented Aug 15, 2024

Closes PROD-2601

Description Of Changes

Fides changes to support consent signal processing in Fidesplus https://github.com/ethyca/fidesplus/pull/1563/

Code Changes

  • Updating the ClientDetail model to support connections in a similar pattern as the existing systems field
  • Adding an optional token_duration_override kwarg to the extract_token_and_load_client function to be able to specify a longer TTL for consent webhook tokens (see the fidesplus PR for more details)
  • Updating the return type for consent override functions from bool to ConsentPropagationStatus to provide a more granular response to the caller instead of just a fired/not-fired boolean
  • Minor cleanup of some parts of the run_consent_request function in the SaaSConnector class

Steps to Confirm

Pre-Merge Checklist

  • All CI Pipelines Succeeded
  • Documentation:
  • Issue Requirements are Met
  • Update CHANGELOG.md
  • If there are any database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!

Copy link

vercel bot commented Aug 15, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
fides-plus-nightly ⬜️ Ignored (Inspect) Visit Preview Aug 27, 2024 3:40am

Copy link

cypress bot commented Aug 15, 2024

fides    Run #9665

Run Properties:  status check passed Passed #9665  •  git commit 3c9263f645 ℹ️: Merge 6de8e7224e37e904aadaab9c54d9672ad49e890d into 619ca8589e90d72494fec95a3074...
Project fides
Branch Review refs/pull/5200/merge
Run status status check passed Passed #9665
Run duration 00m 37s
Commit git commit 3c9263f645 ℹ️: Merge 6de8e7224e37e904aadaab9c54d9672ad49e890d into 619ca8589e90d72494fec95a3074...
Committer Adrian Galvan
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 4
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.
View all changes introduced in this branch ↗︎

Copy link

codecov bot commented Aug 15, 2024

Codecov Report

Attention: Patch coverage is 71.42857% with 8 lines in your changes missing coverage. Please review.

Project coverage is 86.41%. Comparing base (619ca85) to head (6de8e72).
Report is 1 commits behind head on main.

Files Patch % Lines
src/fides/api/service/connectors/saas_connector.py 66.66% 2 Missing and 2 partials ⚠️
...vice/saas_request/saas_request_override_factory.py 25.00% 0 Missing and 3 partials ⚠️
src/fides/api/models/client.py 75.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5200      +/-   ##
==========================================
- Coverage   86.41%   86.41%   -0.01%     
==========================================
  Files         362      362              
  Lines       22781    22792      +11     
  Branches     3058     3060       +2     
==========================================
+ Hits        19687    19695       +8     
- Misses       2537     2538       +1     
- Partials      557      559       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@galvana galvana changed the base branch from main to PROD-1788-track-request-origin August 21, 2024 20:48
@galvana galvana marked this pull request as ready for review August 24, 2024 19:00
@galvana galvana requested a review from a team as a code owner August 24, 2024 19:00
@galvana galvana requested a review from pattisdr August 25, 2024 04:41
Base automatically changed from PROD-1788-track-request-origin to main August 26, 2024 21:27
@@ -654,21 +656,23 @@ def run_consent_request(
identity_data: Dict[str, Any],
session: Session,
) -> bool:
"""Execute a consent request. Return whether the consent request to the third party succeeded.
# pylint: disable=too-many-branches, too-many-statements
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot happening in run_consent_request now, a followup ticket could try to break some of this logic into separate functions -

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed! I thought about doing it during this pass but a follow up ticket sound better https://ethyca.atlassian.net/browse/PROD-2659

Comment on lines +815 to +817
consent_propagation_status = (
ConsentPropagationStatus.missing_data
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems net new - if I'm following this will now cause an exception to be raised on L826 when before it was just ignored? Is this intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We previously set fired = False at the start, and if fired remained False by the end, we assumed data was missing and raised a SkippingConsentPropagation exception with the message "Missing needed values to propagate request". Now, we're setting this explicitly if we get a ValueError when building a consent request and skip_missing_param_values is set to True

except ValueError as exc:
    if consent_request.skip_missing_param_values:

Same behavior as before but now it's more explicit

@galvana galvana merged commit 379154d into main Aug 27, 2024
20 checks passed
@galvana galvana deleted the PROD-2601-updates-for-consent-signal-processing branch August 27, 2024 03:40
Copy link

cypress bot commented Aug 27, 2024

fides    Run #9666

Run Properties:  status check passed Passed #9666  •  git commit 379154dd7b: Updates for consent signal processing (#5200)
Project fides
Branch Review main
Run status status check passed Passed #9666
Run duration 00m 35s
Commit git commit 379154dd7b: Updates for consent signal processing (#5200)
Committer Adrian Galvan
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 4
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.
View all changes introduced in this branch ↗︎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants