Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HUDI-3336][HUDI-FLINK] Support custom hadoop config options for flink #4699

Closed
wants to merge 10 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
Empty file modified .asf.yaml
100644 → 100755
Empty file.
Empty file modified .codecov.yml
100644 → 100755
Empty file.
Empty file modified .github/ISSUE_TEMPLATE/SUPPORT_REQUEST.md
100644 → 100755
Empty file.
Empty file modified .github/ISSUE_TEMPLATE/config.yml
100644 → 100755
Empty file.
Empty file modified .github/PULL_REQUEST_TEMPLATE.md
100644 → 100755
Empty file.
Empty file modified .github/workflows/bot.yml
100644 → 100755
Empty file.
Empty file modified .gitignore
100644 → 100755
Empty file.
Empty file modified .idea/vcs.xml
100644 → 100755
Empty file.
Empty file modified LICENSE
100644 → 100755
Empty file.
Empty file modified NOTICE
100644 → 100755
Empty file.
Empty file modified README.md
100644 → 100755
Empty file.
Empty file modified azure-pipelines.yml
100644 → 100755
Empty file.
Empty file modified conf/hudi-defaults.conf.template
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-flink-bundle_2.11.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-flink-bundle_2.12.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-hadoop-mr-bundle.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-hive-sync-bundle.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-integ-test-bundle.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-kafka-connect-bundle.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-presto-bundle.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-spark-bundle_2.11.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-spark-bundle_2.12.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-spark3-bundle_2.12.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-timeline-server-bundle.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-utilities-bundle_2.11.txt
100644 → 100755
Empty file.
Empty file modified dependencies/hudi-utilities-bundle_2.12.txt
100644 → 100755
Empty file.
Empty file modified doap_HUDI.rdf
100644 → 100755
Empty file.
Empty file modified docker/README.md
100644 → 100755
Empty file.
Empty file modified docker/compose/docker-compose_hadoop284_hive233_spark244.yml
100644 → 100755
Empty file.
Empty file modified docker/compose/hadoop.env
100644 → 100755
Empty file.
Empty file modified docker/demo/compaction-bootstrap.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/compaction.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/config/base.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/dfs-source.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/hoodie-incr.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/hoodie-schema.avsc
100644 → 100755
Empty file.
Empty file modified docker/demo/config/kafka-source.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/log4j.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/schema.avsc
100644 → 100755
Empty file.
Empty file modified docker/demo/config/spark-defaults.conf
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/base.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/compact-test.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/complex-dag-mor.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/complex-source.avsc
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified docker/demo/config/test-suite/insert-overwrite-table.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/insert-overwrite.yaml
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified docker/demo/config/test-suite/mor-async-compact.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/multi-writer-1-ds.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/multi-writer-1.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/multi-writer-2-sds.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/multi-writer-2.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/multi-writer-local-1.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/multi-writer-local-2.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/simple-clustering-hive.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/simple-clustering.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/simple-deltastreamer-hive.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/simple-deltastreamer.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/source.avsc
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/spark-clustering.yaml
100644 → 100755
Empty file.
Empty file.
Empty file modified docker/demo/config/test-suite/spark-long-running.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/spark-medium-clustering.yaml
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/spark-simple.yaml
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified docker/demo/config/test-suite/target.avsc
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified docker/demo/config/test-suite/templates/sanity.yaml.template
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified docker/demo/config/test-suite/test-clustering.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/test-inline-compact.properties
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified docker/demo/config/test-suite/test-metadata.properties
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified docker/demo/config/test-suite/test-nonpartitioned.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/config/test-suite/test.properties
100644 → 100755
Empty file.
Empty file modified docker/demo/data/batch_1.json
100644 → 100755
Empty file.
Empty file modified docker/demo/data/batch_2.json
100644 → 100755
Empty file.
Empty file modified docker/demo/hive-batch1.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/hive-batch2-after-compaction.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/hive-incremental-cow.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/hive-incremental-mor-ro.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/hive-incremental-mor-rt.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/hive-table-check.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/presto-batch1.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/presto-batch2-after-compaction.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/presto-table-check.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/sparksql-batch1.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/sparksql-batch2.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/sparksql-bootstrap-prep-source.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/sparksql-incremental.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/sync-validate.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/trino-batch1.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/trino-batch2-after-compaction.commands
100644 → 100755
Empty file.
Empty file modified docker/demo/trino-table-check.commands
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/base/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/base/entrypoint.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/base/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/base_java11/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/base_java11/entrypoint.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/base_java11/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/datanode/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/datanode/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/datanode/run_dn.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/historyserver/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/historyserver/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/historyserver/run_history.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/hive_base/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/hive_base/conf/beeline-log4j2.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/hive_base/conf/hive-env.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/hive_base/conf/hive-exec-log4j2.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/hive_base/conf/hive-log4j2.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/hive_base/conf/hive-site.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/hive_base/conf/ivysettings.xml
100644 → 100755
Empty file.
Empty file.
Empty file modified docker/hoodie/hadoop/hive_base/entrypoint.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/hive_base/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/hive_base/startup.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/namenode/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/namenode/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/namenode/run_nn.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/prestobase/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/prestobase/etc/catalog/hive.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/prestobase/etc/catalog/jmx.properties
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file modified docker/hoodie/hadoop/prestobase/etc/jvm.config.mustache
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/prestobase/etc/log.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/prestobase/etc/node.properties.mustache
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/prestobase/etc/worker.properties.mustache
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/prestobase/lib/mustache.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/prestobase/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/spark_base/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/spark_base/execute-step.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/spark_base/finish-step.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/spark_base/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/spark_base/wait-for-step.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/sparkadhoc/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/sparkadhoc/adhoc.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/sparkadhoc/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/sparkmaster/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/sparkmaster/master.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/sparkmaster/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/sparkworker/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/sparkworker/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/sparkworker/worker.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinobase/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinobase/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinobase/scripts/trino.sh
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinocoordinator/Dockerfile
100644 → 100755
Empty file.
Empty file.
Empty file modified docker/hoodie/hadoop/trinocoordinator/etc/config.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinocoordinator/etc/jvm.config
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinocoordinator/etc/log.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinocoordinator/etc/node.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinocoordinator/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinoworker/Dockerfile
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinoworker/etc/catalog/hive.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinoworker/etc/config.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinoworker/etc/jvm.config
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinoworker/etc/log.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinoworker/etc/node.properties
100644 → 100755
Empty file.
Empty file modified docker/hoodie/hadoop/trinoworker/pom.xml
100644 → 100755
Empty file.
Empty file modified docker/images/push_to_docker_hub.png
100644 → 100755
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Empty file modified hudi-aws/pom.xml
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified hudi-aws/src/main/java/org/apache/hudi/aws/utils/S3Utils.java
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified hudi-aws/src/test/resources/log4j-surefire.properties
100644 → 100755
Empty file.
Empty file modified hudi-cli/conf/hudi-env.sh
100644 → 100755
Empty file.
Empty file modified hudi-cli/pom.xml
100644 → 100755
Empty file.
Empty file modified hudi-cli/src/main/java/org/apache/hudi/cli/HoodieCLI.java
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified hudi-cli/src/main/java/org/apache/hudi/cli/HoodiePrompt.java
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file modified hudi-cli/src/main/java/org/apache/hudi/cli/Main.java
100644 → 100755
Empty file.
Empty file modified hudi-cli/src/main/java/org/apache/hudi/cli/Table.java
100644 → 100755
Empty file.
Empty file modified hudi-cli/src/main/java/org/apache/hudi/cli/TableHeader.java
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified hudi-cli/src/main/java/org/apache/hudi/cli/utils/HiveUtil.java
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file modified hudi-cli/src/main/java/org/apache/hudi/cli/utils/SparkUtil.java
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file modified hudi-cli/src/main/scala/org/apache/hudi/cli/DeDupeType.scala
100644 → 100755
Empty file.
Empty file.
Empty file modified hudi-cli/src/main/scala/org/apache/hudi/cli/SparkHelpers.scala
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file modified hudi-cli/src/test/resources/clean.properties
100644 → 100755
Empty file.
Empty file modified hudi-cli/src/test/resources/log4j-surefire-quiet.properties
100644 → 100755
Empty file.
Empty file modified hudi-cli/src/test/resources/log4j-surefire.properties
100644 → 100755
Empty file.
Empty file modified hudi-cli/src/test/resources/table-config.properties
100644 → 100755
Empty file.
Empty file modified hudi-client/hudi-client-common/pom.xml
100644 → 100755
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Empty file.
Loading