Sync Tool registers 2 tables, RO and RT Tables #210
Conversation
| //sync a RO table for MOR | ||
| syncHoodieTable(false); | ||
| String originalTableName = cfg.tableName; | ||
| cfg.tableName = cfg.tableName + SUFFIX_REALTIME_TABLE; |
There was a problem hiding this comment.
Another way to do this is to add an optional config parameter to HiveSyncConfig (only for MOR type of tables) and then refactor parts of HiveSyncTool and HiveHoodieClient to use it conditionally for MOR table types. I feel this keeps conditional clauses away from HiveHoodieClient,HiveSyncTool.
There was a problem hiding this comment.
Can you leave a todo - its always best to have a possible config
| //sync a RO table for MOR | ||
| syncHoodieTable(false); | ||
| String originalTableName = cfg.tableName; | ||
| cfg.tableName = cfg.tableName + SUFFIX_REALTIME_TABLE; |
There was a problem hiding this comment.
Can you leave a todo - its always best to have a possible config
| switch(hoodieHiveClient.getTableType()) { | ||
| case COPY_ON_WRITE: | ||
| syncHoodieTable(false); | ||
| hoodieHiveClient.close(); |
There was a problem hiding this comment.
Do this once outside the switch
There was a problem hiding this comment.
I've added one to the default case. There should be no scenario where we reach outside the switch without an exception (in which case it doesn't there).
There was a problem hiding this comment.
I meant why repeat this on every case instead of doing it once outside the switch once.
There was a problem hiding this comment.
Makes sense, should've been obvious.
| // TODO - RO Table for MOR only after major compaction (UnboundedCompaction is default for now) | ||
| hoodieHiveClient.createTable(schema, HoodieInputFormat.class.getName(), | ||
| MapredParquetOutputFormat.class.getName(), ParquetHiveSerDe.class.getName()); | ||
| } else { | ||
| // create RT Table |
n3nash
force-pushed
the
hive_register_2_tables
branch
from
4488228 to
7fe8623
Compare
n3nash
force-pushed
the
hive_register_2_tables
branch
from
7fe8623 to
36a1947
Compare
|
@prazanna I've addressed all the comments. |
No description provided.