Skip to content

Commit

Permalink
[doc]modified the spark-load doc (apache#9605)
Browse files Browse the repository at this point in the history
  • Loading branch information
zy-kkk authored and starocean999 committed May 19, 2022
1 parent d73d5eb commit 5d7ecc8
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 1 deletion.
4 changes: 4 additions & 0 deletions docs/en/data-operate/import/import-way/spark-load-manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -627,6 +627,10 @@ If `spark_resource_path` is not set correctly. An error `file XXX/jars/spark-2x.

If `yarn_client_path` is not set correctly. An error `yarn client does not exist in path: XXX/yarn-client/hadoop/bin/yarn` will be reported.

* When using spark load, the `JAVA_HOME` environment variable is no set in the `hadoop-config.sh` on the yarn clinet.

If the `JAVA_HOME` environment variable is not set, the error `yarn application kill failed. app id: xxx, load job id: xxx, msg: which: no xxx/lib/yarn-client/hadoop/bin/yarn in ((null)) Error: JAVA_HOME is not set and could not be found` will be reported.

## More Help

For more detailed syntax used by **Spark Load**, you can enter `HELP SPARK LOAD` on the Mysql client command line for more help.
Original file line number Diff line number Diff line change
Expand Up @@ -559,7 +559,7 @@ LoadFinishTime: 2019-07-27 11:50:16

## 常见问题

- 使用 Spark Load 时没有在 spark 客户端的 spark-env.sh 配置 `HADOOP_CONF_DIR` 环境变量。
- 使用 Spark Load 时没有在 spark 客户端的 `spark-env.sh` 配置 `HADOOP_CONF_DIR` 环境变量。

如果 `HADOOP_CONF_DIR` 环境变量没有设置,会报 `When running with master 'yarn' either HADOOP_CONF_DIR or YARN_CONF_DIR must be set in the environment.` 错误。

Expand All @@ -575,6 +575,10 @@ LoadFinishTime: 2019-07-27 11:50:16

如果 `yarn_client_path `没有设置正确,会报 `yarn client does not exist in path: xxx/yarn-client/hadoop/bin/yarn` 错误

- 使用Spark load 时没有在 yarn 客户端的 `hadoop-config.sh` 配置 `JAVA_HOME` 环境变量。

如果 `JAVA_HOME` 环境变量没有设置,会报 `yarn application kill failed. app id: xxx, load job id: xxx, msg: which: no xxx/lib/yarn-client/hadoop/bin/yarn in ((null)) Error: JAVA_HOME is not set and could not be found` 错误

## 更多帮助

关于**Spark Load** 使用的更多详细语法,可以在Mysql客户端命令行下输入 `HELP SPARK LOAD` 获取更多帮助信息。
Expand Down

0 comments on commit 5d7ecc8

Please sign in to comment.