Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding property ID to privacy preference history #4886

Merged
merged 15 commits into from
May 20, 2024

Conversation

galvana
Copy link
Contributor

@galvana galvana commented May 13, 2024

Closes PROD-2037

Code Changes

  • Adding optional property_id to PrivacyPreferenceHistory model

Steps to Confirm

Pre-Merge Checklist

  • All CI Pipelines Succeeded
  • Issue Requirements are Met
  • Update CHANGELOG.md

Copy link

vercel bot commented May 13, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
fides-plus-nightly ⬜️ Ignored (Inspect) Visit Preview May 20, 2024 11:21pm

Copy link

cypress bot commented May 13, 2024

Passing run #7816 ↗︎

0 4 0 0 Flakiness 0
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.

Details:

Merge 2c9433e into ab33900...
Project: fides Commit: 4d76d9ed2e ℹ️
Status: Passed Duration: 00:34 💡
Started: May 20, 2024 10:35 PM Ended: May 20, 2024 10:36 PM

Review all test suite changes for PR #4886 ↗︎

Copy link

codecov bot commented May 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.75%. Comparing base (ab33900) to head (2c9433e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4886      +/-   ##
==========================================
- Coverage   86.75%   86.75%   -0.01%     
==========================================
  Files         347      347              
  Lines       20965    20966       +1     
  Branches     2741     2741              
==========================================
  Hits        18189    18189              
  Misses       2297     2297              
- Partials      479      480       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@galvana galvana marked this pull request as ready for review May 13, 2024 18:54
@galvana galvana requested a review from pattisdr May 13, 2024 18:57
@@ -388,6 +388,8 @@ def __tablename__(self) -> str:
MutableDict.as_mutable(JSONB)
) # Dict of TCF attributes saved, for a TCF notice

property_id = Column(String, index=True, nullable=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, this is the same as plus_property.id right? Just trying to think through if this should be a FK. We are enforcing other resources like the privacy notice, privacy notice history, etc. be proper FK's.

Cons are that this is a large table for some customers, adding a FK is tricky. It could be added with something like a "NOT VALID" at first and then issuing a separate query to validate the constraint separately.

It might not be worth that though, a string could be fine too, just wanted to surface

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! Same thing, that was my intention but I forgot to add the foreign key constraint 😵‍💫. I set it to be nullable since it's not guaranteed that the associated privacy experience will be retrieved by property ID, which is the main bit of info we're tracking with this new addition. I also set ondelete="SET NULL" since we want to be able to delete properties freely and still keep these historical records around.

I didn't know about "NOT VALID" in Postgres so thanks for sharing! How would that apply here? From my understanding it allows us to delay the constraint validation and create the constraint more quickly. Would this optimization still be needed if every row has a null property_id?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussed in person, we'll revert back to a string to preserve the history in case the property is deleted in the future.

@galvana galvana requested a review from pattisdr May 14, 2024 20:07
Comment on lines 23 to 28
op.create_index(
op.f("ix_privacypreferencehistory_property_id"),
"privacypreferencehistory",
["property_id"],
unique=False,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@galvana galvana requested a review from pattisdr May 20, 2024 20:32
Comment on lines 330 to 334
property_id = Column(
String,
index=True,
nullable=True,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a quick model-level test on this field? There's an existing file tests/ops/models/test_privacy_preference.py

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, we could define this field on the ConsentReportingMixinV2 in one place I believe, it's shared between PrivacyPreferenceHistory and ServedNoticeHistory

Copy link
Contributor

@pattisdr pattisdr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved pending a couple of requested changes to add a model-level test, and to consolidate the field definitions on the ConsentReportingMixinV2

@galvana galvana merged commit 6c56d38 into main May 20, 2024
14 checks passed
@galvana galvana deleted the PROD-2037-store-property-ID-alongside-consent-preference branch May 20, 2024 23:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants