HJ-58: migrate dataset field names

[thingscouldbeworse]

Closes #HJ-58

Description Of Changes

Adds an endpoint that can be called which repairs field names in datasets from the old dot-separated sub field names to the new, trimmed, source names

Code Changes

• add /datasets/clean, and endpoint which extracts all existing datasets, changes the field names in the collections from parent.child.grandchild to grandchild, updates the ctl_datasets row with the new collection

Steps to Confirm

• add any dataset you wish through the UI. Add some nested fields with names with dots, ie name.sub_name
• hit the endpoint through swagger
• confirm the names are now changed to just sub_name

Pre-Merge Checklist

• All CI Pipelines Succeeded
• Documentation:
  • documentation complete, PR opened in fidesdocs
  • documentation issue created in fidesdocs
  • if there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
• Issue Requirements are Met
• Relevant Follow-Up Issues Created
• Update CHANGELOG.md
• For API changes, the Postman collection has been updated
• If there are any database migrations:
  • Ensure that your downrev is up to date with the latest revision on main
  • Ensure that your downgrade() migration is correct and works
    • If a downgrade migration is not possible for this change, please call this out in the PR description!

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
fides-plus-nightly	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 6, 2024 9:04pm

[cypress]

fides    Run #10863

Run Properties:   Passed #10863  •  67ff4a61f8 ℹ️: Merge e78b2fc2c8b13115b94a8a99e80c7526acdfda6f into 55d34540d154f44c642c0da954c6...

Project	fides
Branch Review	refs/pull/5461/merge
Run status	Passed #10863
Run duration	00m 39s
Commit	67ff4a61f8 ℹ️: Merge e78b2fc2c8b13115b94a8a99e80c7526acdfda6f into 55d34540d154f44c642c0da954c6...
Committer	Kirk Hardy
View all properties for this run ↗︎

---

Test results
Failures	0
Flaky	0
Pending	0
Skipped	0
Passing	4
⚠️ You've recorded test results over your free plan limit. Upgrade your plan to view test results.
View all changes introduced in this branch ↗︎

[codecov]

Codecov Report

Attention: Patch coverage is 21.21212% with 26 lines in your changes missing coverage. Please review.

Project coverage is 85.33%. Comparing base (5ca600a) to head (c41db1b).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...rc/fides/api/api/v1/endpoints/dataset_endpoints.py	21.21%	26 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5461      +/-   ##
==========================================
- Coverage   85.42%   85.33%   -0.09%     
==========================================
  Files         385      385              
  Lines       24152    24185      +33     
  Branches     2630     2635       +5     
==========================================
+ Hits        20631    20638       +7     
- Misses       2967     2993      +26     
  Partials      554      554              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[erosselli]

Tested manually and it worked

[cypress]

fides    Run #10862

Run Properties:   Passed #10862  •  66ec9d2d89: HJ-58: migrate dataset field names (#5461)

Project	fides
Branch Review	main
Run status	Passed #10862
Run duration	00m 37s
Commit	66ec9d2d89: HJ-58: migrate dataset field names (#5461)
Committer	Kirk Hardy
View all properties for this run ↗︎

---

Test results
Failures	0
Flaky	0
Pending	0
Skipped	0
Passing	4
⚠️ You've recorded test results over your free plan limit. Upgrade your plan to view test results.
View all changes introduced in this branch ↗︎