You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Upgraded to hudi 0.12 here (from 0.10) and it seems some things changed to the HiveSyncTool, more specifically to HiveSyncConfig.java
Where before there were defaults for some Boolean parameters, this is now no longer the case. As an example:
@Parameter(names = {"--spark-datasource"}, description = "Whether sync this table as spark data source table.")
public Boolean syncAsSparkDataSourceTable;
used to be
@Parameter(names = {"--spark-datasource"}, description = "Whether sync this table as spark data source table.")
public Boolean syncAsSparkDataSourceTable = true;
When I try to run the synctool commandline, I now get NPE's as soon as I include one of these Boolean parameters.
I found cbeust/jcommander#447
This does not appear to be fixed in later JCommander releases at first sight (did a quick test overriding the dependencies).
Haven't tested yet whether switching to boolean as suggested in that issue fixes it. Reverting to default values as before also has the issue cbeust/jcommander#378 (so it then just flips the set default which is weird). Specifying an arity 1 so the value has to be set is maybe another option (https://jcommander.org/).
To Reproduce
Steps to reproduce the behavior:
Run HiveSyncTool main and specify one of the boolean parameters; e.g. --spark-datasource, --ignore-exceptions...
a NullPointerException will be thrown
Expected behavior
Should run fine
Environment Description
Hudi version : 0.12
Stacktrace
java.lang.NullPointerException:
[info] at org.apache.hudi.com.beust.jcommander.JCommander.parseValues(JCommander.java:718)
[info] at org.apache.hudi.com.beust.jcommander.JCommander.parse(JCommander.java:340)
[info] at org.apache.hudi.com.beust.jcommander.JCommander.parse(JCommander.java:319)
[info] at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:353)
The text was updated successfully, but these errors were encountered:
Describe the problem you faced
Upgraded to hudi 0.12 here (from 0.10) and it seems some things changed to the HiveSyncTool, more specifically to HiveSyncConfig.java
Where before there were defaults for some Boolean parameters, this is now no longer the case. As an example:
used to be
When I try to run the synctool commandline, I now get NPE's as soon as I include one of these Boolean parameters.
I found cbeust/jcommander#447
This does not appear to be fixed in later JCommander releases at first sight (did a quick test overriding the dependencies).
Haven't tested yet whether switching to boolean as suggested in that issue fixes it. Reverting to default values as before also has the issue cbeust/jcommander#378 (so it then just flips the set default which is weird). Specifying an arity 1 so the value has to be set is maybe another option (https://jcommander.org/).
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Should run fine
Environment Description
Stacktrace
The text was updated successfully, but these errors were encountered: