Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(platform): Schema version history timeline. #4252

Merged
merged 45 commits into from
Feb 25, 2022

Conversation

rslanka
Copy link
Contributor

@rslanka rslanka commented Feb 25, 2022

Adds the schema version history feature.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable)

rslanka and others added 30 commits February 24, 2022 19:42
adding new classes for computing timeline

add json patch lib

add initial controller

fix test

reformat code

improve test

timeline command

v1

detect double encoding

add start and end

initial docs & fix endpoint

move to openapi

split openapi to different servlet

fix docs showing up issue

Implements SchemaDiffer for ordinal based schemas.

Update tests

WIP: Don't checkin non-ordinal based impl.

Implements diffing logic for non-ordinal based schemas in the SchemaDiffer.

unwinding settings change

fix element id

add logic to convert strings to enum

Primary key constraint change events.

Added logic for assigning computed semantic version to transactions.

add apis to frontend dropdown

Minor refactoring, renaming/rearranging of fields.

add exceptional logic to schemadiffer and example files

Move transaction id out of ChangeTransaction.

Fix NPE if both base and target fields have no description.

obey rawdiffrequested, update how jsonpatch appears as schema in swagger, remove mock service, update CLI

Generate EXCEPTIONAL change events when schema are not compatible.

Revert the RESTLI based implementaion: "Schema Version Change APIs + basic differ implementation."
This reverts commit def13e5528701fffa2510ed5ee792323a65f850b.

tags based differ should be basicdiffer

small fixes and improvements

fixing lint

Use V1 style field paths in the description

Fix issue with version field in v1 fieldPath conversion logic.

fix test to not assert :(

adding colored output

Fix mypy issue with termcolor.

remove unused files

rename element

Use urns to populate elementId & target.

Replace DatasetFieldUrns with schemaField urns.

improve rendering

final fix
@rslanka rslanka changed the title feat(schema): Schema version history. feat(platform): Schema version history timeline. Feb 25, 2022
@github-actions
Copy link

github-actions bot commented Feb 25, 2022

Unit Test Results (build & test)

  71 files  +1    71 suites  +1   11m 3s ⏱️ -26s
618 tests +7  559 ✔️ +7  59 💤 ±0  0 ±0 

Results for commit d1a5e84. ± Comparison against base commit 3a0fe44.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Feb 25, 2022

Unit Test Results (metadata ingestion)

       5 files         5 suites   44m 6s ⏱️
   337 tests    337 ✔️   0 💤 0
1 532 runs  1 501 ✔️ 31 💤 0

Results for commit d1a5e84.

♻️ This comment has been updated with latest results.

Copy link
Contributor

@shirshanka shirshanka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@shirshanka shirshanka merged commit 1ac20ee into datahub-project:master Feb 25, 2022
maggiehays pushed a commit to maggiehays/datahub that referenced this pull request Aug 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants