Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor(sdk/patch): improve patch implementation internals #12253

Merged
merged 11 commits into from
Jan 2, 2025
Merged

refactor(sdk/patch): improve patch implementation internals #12253

merged 11 commits into from
Jan 2, 2025

Conversation

hsheth2
Copy link
Collaborator

@hsheth2 hsheth2 commented Dec 31, 2024 •
edited
Loading

  • Replace patch path strings with proper tuples. This ensures that urns are always quoted properly, and we cannot make simple errors with it.
  • Replaces OwnershipPatchHelper and similar with mixin classes. This will (1) significantly reduce the amount of code duplication across our patch codebase and (2) make it much easier to add support for new entity types.
  • This causes no changes to the interface or tests - all user-facing functionality is retained.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added the ingestion PR or Issue related to the ingestion of metadata label Dec 31, 2024
@hsheth2 hsheth2 changed the title replace all patch strings with paths for automatic quoting refactor(sdk/patch): improve patch implementation internals Dec 31, 2024
@github-actions github-actions bot requested a deployment to datahub-wheels (Preview) December 31, 2024 20:25 Abandoned
@datahub-cyborg datahub-cyborg bot added the needs-review Label for PRs that need review from a maintainer. label Dec 31, 2024
mayurinehate
mayurinehate approved these changes Jan 2, 2025
View reviewed changes
Copy link
Collaborator

@mayurinehate mayurinehate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty clean. It would be great if we can add unit tests for each of these patch helpers . Maybe some AI magic followed by manual review :)

metadata-ingestion/src/datahub/specific/dataset.py Outdated
@@ -181,10 +176,13 @@ def get_fine_grained_key(
@classmethod
def quote_fine_grained_path(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably rename this method now that this is not responsible for quoting ?

@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed needs-review Label for PRs that need review from a maintainer. labels Jan 2, 2025
@hsheth2
Copy link
Collaborator Author

hsheth2 commented Jan 2, 2025 •
edited
Loading

@mayurinehate most of this functionality should be tested via the dataset patch builder tests.

@hsheth2 hsheth2 added merge-pending-ci A PR that has passed review and should be merged once CI is green. and removed pending-submitter-merge labels Jan 2, 2025
@hsheth2 hsheth2 merged commit f396d8d into master Jan 2, 2025
81 of 108 checks passed
@hsheth2 hsheth2 deleted the update-patch branch January 2, 2025 20:36
brock-acryl pushed a commit to brock-acryl/datahub that referenced this pull request Jan 30, 2025
@hsheth2 @chakru-r
refactor(sdk/patch): improve patch implementation internals (datahub-…
b90dd4e 
…project#12253)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mayurinehate mayurinehate mayurinehate approved these changes

Assignees
No one assigned
Labels
ingestion PR or Issue related to the ingestion of metadata merge-pending-ci A PR that has passed review and should be merged once CI is green.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@hsheth2 @mayurinehate