Updating privacy request search endpoint

[galvana]

Closes PROD-2124

Description Of Changes

Marked the existing GET /privacy-request as deprecated in favor of a new POST /privacy-request/search endpoint. We needed to specify custom identities and custom privacy request fields but these inputs are a combination of identity/custom field name or and value. I first thought of supporting dynamic query params but that didn't work out since we need to make a distinction if the new query param is an identity or a custom field lookup.

I also made the decision to include the identity or custom field name for the search to be more specific. The current GET /privacy-request endpoint just matches on the identity value without considering which identity type the result matched on.

The new endpoint also removes the single string query param of identity in favor of an identities value in the POST body.

{
  "identities": {
    "email": "[email protected]"
  }
}

The external_id field is still supported as a standalone field, for backwards compatibility, even though we support external_id as a default identity type as well

{
  "identities": {
    "external_id": "ext-123"
  }
}

Code Changes

• New POST /privacy-request/search endpoint and PrivacyRequestFilter schema to validate the payload.

Steps to Confirm

• Set FIDES__EXECUTION__ALLOW_CUSTOM_PRIVACY_REQUEST_FIELD_COLLECTION=true in your .env file to enable custom privacy request field collection
• Create a new privacy request with custom identities and custom privacy request fields

curl -X 'POST' \
  'http://localhost:8080/api/v1/privacy-request' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '[
  {
    "policy_key": "default_access_policy",
    "identity": {
      "email": "[email protected]",
      "loyalty_id": {
        "label": "Loyalty ID",
        "value": "CH-1"
      }
    },
    "custom_privacy_request_fields": {
      "site_id": {
        "label": "Site ID",
        "value": "site-123"
      }
    }
  }
]'

• Use the new endpoint to search by the new fields

curl -X 'POST' \
  'http://localhost:8080/api/v1/privacy-request/search?page=1&size=50' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer REDACTED' \
  -H 'Content-Type: application/json' \
  -d '{
  "identities": {
    "loyalty_id": "CH-1"
  }
}'

curl -X 'POST' \
  'http://localhost:8080/api/v1/privacy-request/search?page=1&size=50' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer REDACTED' \
  -H 'Content-Type: application/json' \
  -d '{
  "custom_privacy_request_fields": {
    "site_id": "site-123"
  }
}'

Pre-Merge Checklist

• All CI Pipelines Succeeded
• Issue Requirements are Met
• Update CHANGELOG.md

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
fides-plus-nightly	⬜️ Ignored (Inspect)	Visit Preview		Jun 18, 2024 2:39am

[cypress]

Passing run #8359 ↗︎

0	4	0	0	0
⚠️ You've recorded test results over your free plan limit. Upgrade your plan to view test results.

Details:

Merge 459a636 into d864095...
Project: fides	Commit: 4d22dcad33 ℹ️
Status: Passed	Duration: 00:33 💡
Started: Jun 18, 2024 2:51 AM	Ended: Jun 18, 2024 2:51 AM

Review all test suite changes for PR #4987 ↗︎

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.54%. Comparing base (3400848) to head (459a636).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4987      +/-   ##
==========================================
+ Coverage   86.51%   86.54%   +0.02%     
==========================================
  Files         351      351              
  Lines       21693    21738      +45     
  Branches     2872     2878       +6     
==========================================
+ Hits        18768    18813      +45     
  Misses       2420     2420              
  Partials      505      505              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pattisdr]

@galvana can you make sure unit tests are running over here before I start a review?

[pattisdr]

Thanks for helping with the numpy changes and getting tests passing on this branch @galvana 👍 starting review now -

[pattisdr]

Just some additional documentation requested if you agree with me there - really nice PR

[cypress]

Passing run #8360 ↗︎

0	4	0	0	0
⚠️ You've recorded test results over your free plan limit. Upgrade your plan to view test results.

Details:

Updating privacy request search endpoint (#4987)
Project: fides	Commit: 24e2539607
Status: Passed	Duration: 00:34 💡
Started: Jun 18, 2024 3:58 AM	Ended: Jun 18, 2024 3:59 AM

Review all test suite changes for PR #4987 ↗︎