Surface AC Systems in Vendor Consents section of the TCF Overlay #4266

pattisdr · 2023-10-12T16:40:02Z

Closes https://github.com/ethyca/fidesplus/issues/1155

Description Of Changes

AC systems should show up in the TCF overlay and need vendor consent toggles. They are not required to have privacy declarations though, which currently drives what is in the TCF overlay.

Code Changes

Surface AC Systems (systems whose vendor_id starts with an "ac.*" prefix) in the TCF overlay under the Vendor Consents section, even if it has no privacy declarations.
Separately update the "relevant systems" when vendor consent preferences are saved to include ac systems in the lookup

Steps to Confirm

Set tcf_enabled=true in fides.toml
Add a system, setting vendor_id to be "ac.100"
GET Privacy Experiences from the EEA

curl -X 'GET' \
  'http://localhost:8080/api/v1/privacy-experience?show_disabled=true&region=EEA&systems_applicable=false&include_gvl=false&include_meta=true&page=1&size=50' \
  -H 'accept: application/json'

TCF Vendor consents section should have the AC system. There won't be any purpose consents unless it has privacy declarations with relevant purposes and consent legal basis:

 "tcf_vendor_consents": [
        {
          "id": "gvl.8",
          "has_vendor_id": true,
          "name": "Emerse",
          "description": "Emerse Sverige AB is a provider of programmatic advertising solutions, offering advertisers and publishers tools to manage and optimize their digital ad campaigns.",
          "default_preference": "opt_out",
          "current_preference": null,
          "outdated_preference": null,
          "current_served": null,
          "outdated_served": null,
          "purpose_consents": [
            {
              "id": 1,
              "name": "Store and/or access information on a device"
            },
            {
              "id": 3,
              "name": "Create profiles for personalised advertising"
            },
            {
              "id": 4,
              "name": "Use profiles to select personalised advertising"
            }
          ]
        },
        {
          "id": "ac.100",
          "has_vendor_id": true,
          "name": "test",
          "description": "",
          "default_preference": "opt_out",
          "current_preference": null,
          "outdated_preference": null,
          "current_served": null,
          "outdated_served": null,
          "purpose_consents": []
        }
      ]

Pre-Merge Checklist

All CI Pipelines Succeeded
Documentation:
- documentation complete, PR opened in fidesdocs
- documentation issue created in fidesdocs
Issue Requirements are Met
Relevant Follow-Up Issues Created
Update CHANGELOG.md
For API changes, the Postman collection has been updated

src/fides/api/util/tcf/tcf_experience_contents.py

cypress · 2023-10-12T16:57:52Z

Passing run #4678 ↗︎

0	4	0	0	0
⚠️ You've recorded test results over your free plan limit. Upgrade your plan to view test results.

Details:

Merge `889e821` into `a862790`...
Project: fides	Commit: `0efe76ae3f ℹ️`
Status: Passed	Duration: 00:50 💡
Started: Oct 18, 2023 5:32 AM	Ended: Oct 18, 2023 5:33 AM

Review all test suite changes for PR #4266 ↗︎

codecov · 2023-10-12T17:29:48Z

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (e8bcfcc) 87.75% compared to head (889e821) 87.76%.
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4266      +/-   ##
==========================================
+ Coverage   87.75%   87.76%   +0.01%     
==========================================
  Files         331      331              
  Lines       20907    20922      +15     
  Branches     2711     2714       +3     
==========================================
+ Hits        18346    18362      +16     
+ Misses       2094     2093       -1     
  Partials      467      467

Files	Coverage Δ
src/fides/api/util/tcf/tc_model.py	`94.96% <100.00%> (-0.04%)`	⬇️
src/fides/api/util/tcf/tc_string.py	`100.00% <100.00%> (ø)`
src/fides/api/util/tcf/tcf_experience_contents.py	`94.16% <100.00%> (+0.41%)`	⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

allisonking

this looks great—very clean and easy to follow, well done! will test it out with the FE now to see if things Just Work and report back

src/fides/api/util/tcf/tcf_experience_contents.py

allisonking · 2023-10-13T19:08:01Z

ah I guess this will also need the prefix change before I can test E2E with local compass

pattisdr · 2023-10-13T19:16:41Z

I'll start on that now!

…fix) in the TCF overlay under the Vendor Consents section, even if it has no privacy declarations. - Separately update the "relevant systems" when vendor consent preferences are saved to include ac systems in the lookup -

- Dry up defining AC and GVL prefixes.

pattisdr · 2023-10-17T17:56:05Z

Rebased onto main due to messy diff when this was originally based off of the compass integration branch. @adamsachs

adamsachs

the changes look great here @pattisdr, this is a lot less painful than i expected, which owes a lot to your clean implementation! just had some minor nits, none of which i'd consider blocking - but would be curious to get your thoughts on them.

i didn't get a chance to smoke test, but if you've done some manual testing, then i think that's sufficient since the changes are straightforward enough. i can circle back and do some testing though if you'd like that extra confirmation!

src/fides/api/util/tcf/tcf_experience_contents.py

adamsachs · 2023-10-17T20:17:53Z

src/fides/api/util/tcf/tcf_experience_contents.py

+    if not tcf_vendor_component_type == TCFVendorConsentRecord:
+        return
+
+    if not system_identifier.startswith(AC_PREFIX):
+        return
+
+    if system_identifier in vendor_map:
+        return


out of curiosity - do you think this reads easier as 3 separate conditionals rather than joining them in a single conditional with ors? just caught my eye since it's not the way i would have written it, but i'm guessing this was deliberate so i'm curious to hear! no need to adjust obviously.

I do think it's easier to parse this way 😎

src/fides/api/util/tcf/tcf_experience_contents.py

tests/fixtures/application_fixtures.py

tests/ops/util/test_tcf_experience_contents.py

- Update docstring - Add test for experience contents with an AC system with a valid privacy declaration

pattisdr · 2023-10-18T12:07:22Z

Thanks for the review Adam - I always do manual testing! I'll go ahead and merge and get the follow-up PR up that is building AC strings on the backend, building off of this work -

pattisdr commented Oct 12, 2023

View reviewed changes

src/fides/api/util/tcf/tcf_experience_contents.py Show resolved Hide resolved

pattisdr requested review from allisonking and adamsachs October 13, 2023 13:49

NevilleS mentioned this pull request Oct 13, 2023

Support an option to prevent dismissing the Fides.js consent banner #4133

Closed

allisonking reviewed Oct 13, 2023

View reviewed changes

src/fides/api/util/tcf/tcf_experience_contents.py Show resolved Hide resolved

src/fides/api/util/tcf/tcf_experience_contents.py Show resolved Hide resolved

pattisdr added the do not merge Please don't merge yet, bad things will happen if you do label Oct 13, 2023

Base automatically changed from integration/compass-1.0.0 to main October 16, 2023 13:53

pattisdr added 3 commits October 17, 2023 12:48

Update ac. prefix to be gacp. prefix.

a640344

- Dry up defining AC and GVL prefixes.

Update changelog.

035dfce

pattisdr force-pushed the fidesplus_1155_ac_vendors_in_overlay_PROD-1164 branch from 5d1e697 to 035dfce Compare October 17, 2023 17:55

adamsachs approved these changes Oct 17, 2023

View reviewed changes

Don't allow AC vendors to be in system identifiers just in case.

ee0661c

- Update docstring - Add test for experience contents with an AC system with a valid privacy declaration

pattisdr removed the do not merge Please don't merge yet, bad things will happen if you do label Oct 18, 2023

Fix missed key in test.

889e821

pattisdr merged commit a9b5950 into main Oct 18, 2023
39 of 40 checks passed

pattisdr deleted the fidesplus_1155_ac_vendors_in_overlay_PROD-1164 branch October 18, 2023 12:07

eastandwestwind pushed a commit that referenced this pull request Oct 18, 2023

Surface AC Systems in Vendor Consents section of the TCF Overlay (#4266)

0546d1c

pattisdr mentioned this pull request Oct 19, 2023

Parse ac string from fides string #4308

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Surface AC Systems in Vendor Consents section of the TCF Overlay #4266

Surface AC Systems in Vendor Consents section of the TCF Overlay #4266

pattisdr commented Oct 12, 2023 •

edited

Loading

cypress bot commented Oct 12, 2023 •

edited

Loading

codecov bot commented Oct 12, 2023 •

edited

Loading

allisonking left a comment

allisonking commented Oct 13, 2023

pattisdr commented Oct 13, 2023

pattisdr commented Oct 17, 2023

adamsachs left a comment

adamsachs Oct 17, 2023

pattisdr Oct 18, 2023

pattisdr commented Oct 18, 2023

Surface AC Systems in Vendor Consents section of the TCF Overlay #4266

Surface AC Systems in Vendor Consents section of the TCF Overlay #4266

Conversation

pattisdr commented Oct 12, 2023 • edited Loading

Description Of Changes

Code Changes

Steps to Confirm

Pre-Merge Checklist

cypress bot commented Oct 12, 2023 • edited Loading

Passing run #4678 ↗︎

Review all test suite changes for PR #4266 ↗︎

codecov bot commented Oct 12, 2023 • edited Loading

Codecov Report

allisonking left a comment

Choose a reason for hiding this comment

allisonking commented Oct 13, 2023

pattisdr commented Oct 13, 2023

pattisdr commented Oct 17, 2023

adamsachs left a comment

Choose a reason for hiding this comment

adamsachs Oct 17, 2023

Choose a reason for hiding this comment

pattisdr Oct 18, 2023

Choose a reason for hiding this comment

pattisdr commented Oct 18, 2023

pattisdr commented Oct 12, 2023 •

edited

Loading

cypress bot commented Oct 12, 2023 •

edited

Loading

codecov bot commented Oct 12, 2023 •

edited

Loading