[HUDI-3922] parse record key + partition path config consistently between keygens and HiveSync #5520
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the purpose of the pull request
hive-sync and the key generators (such as ComplexKeyGenerator) handles the degenerate case (empty string) differently.
Previously this was merely a configuration annoyance, but now it causes issues (see #5101).
We parse configs consistently between the various key generators and HiveSyncConfig
Brief change log
TypedProperties#getStringList
(this causes HoodieSyncConfig to treathoodie.datasource.hive_sync.partition_fields=
the same as not specifying it)PARTITIONPATH_FIELD_NAME
similarly in the various key generatorsRECORDKEY_FIELD_NAME
similarly, and allow leaving not specifyingRECORDKEY_FIELD_NAME
to mean the default.Verify this pull request
This PR modifies the handling of degenerate configurations. The affected test cases are updated to match the new behaviour.
Committer checklist
Has a corresponding JIRA in PR title & commit
Commit message is descriptive of the change
CI is green
Necessary doc changes done or have another open PR
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.