Dev 1.9.0 code fix #627

v-kkhuang · 2024-10-29T07:13:12Z

What is the purpose of the change

EngineConn-Core defines the the abstractions and interfaces of the EngineConn core functions.
The Engine Service in Linkis 0.x is refactored, EngineConn will handle the engine connection
and session management.

Related issues/PRs

Related issues: close #590 close #591
Related pr:#591

Brief change log

Define the core abstraction and interfaces of the EngineConn Factory;
Define the core abstraction and interfaces of Executor Manager.

Checklist

I have read the Contributing Guidelines on pull requests.
I have explained the need for this PR and the problem it solves
I have explained the changes or the new features added to this PR
I have added tests corresponding to this change
I have updated the documentation to reflect this change
I have verified that this change is backward compatible (If not, please discuss on the Linkis mailing list first)
If this is a code change: I have written unit tests to fully verify the new behavior.

…-webank-merge

* ai generate code init * 人工修改代码 * 人工修改提交

* 代码由AI自动生成接口和AI插件生成 * 代码由AI自动生成接口和AI插件生成 * 代码由人工修改 --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

* 代码由AI自动生成接口和AI插件生成 * 代码由AI自动生成接口和AI插件生成 * 代码由人工修改 * bug fix * bug fix --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

* 使用ai生成单元测试案例 * 人工修改单元测试 * 人工修改单元测试 * add instance info

…-webank-merge # Conflicts: # linkis-computation-governance/linkis-manager/linkis-application-manager/src/main/scala/org/apache/linkis/manager/am/conf/AMConfiguration.scala

…-webank

…615) * Added StarRocks task timeout alarm and timeout kill function * Added StarRocks task timeout alarm and timeout kill function --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

-Change DecimalType to class, supporting different precision and decimal places -Modify the dataType recognition logic to support various representations of the decimal type -Update Excel write logic to correctly handle different decimal types Co-authored-by: “v_kkhuang” <“[email protected]”>

* Data source password encryption enhancement Introduce AES encryption mechanism to enhance the security of data source passwords. This submission includes the implementation of encrypting and decrypting data source passwords to ensure password security during storage and transmission. * Using Error Exception to Handle Exceptions in AESUtils * 加密和密码管理改进 - 更新了AESUtils中的错误异常消息，以符合代码规范。 - 在ConfigurationRestfulApi中，对特定配置的密码进行了加密处理，以增强安全性。 - 修改了DataSourceCoreRestfulApi中的注释，并对密码字段的加密处理进行了优化。 - 在NebulaEngineConnExecutor中，密码现在使用AES进行解密，以确保数据安全性。 * Code optimization, adding data source encryption switch * Code optimization, adding data source encryption switch * Update AESUtils.java update default key --------- Co-authored-by: “v_kkhuang” <“[email protected]”> Co-authored-by: Casion <[email protected]>

Co-authored-by: “v_kkhuang” <“[email protected]”>

…inspection (#617) * 添加部门级别的结果集限制配置和检查 - 在 Configuration.scala 中添加 JOB_RESULT_DEPARTMENT_LIMIT 变量，用于配置部门结果集限制- 实现 canResultSetByDepartment 方法，根据部门 ID 判断是否有结果集权限 - 在 QueryRestfulApi.java 中调用 canResultSetByDepartment 方法，将结果加入 API响应 * Update Configuration.scala update default department id --------- Co-authored-by: “v_kkhuang” <“[email protected]”> Co-authored-by: Casion <[email protected]>

…-webank

* fix hive engine error * Security work order repair * sync ddl --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

…-webank

Co-authored-by: “v_kkhuang” <“[email protected]”>

* fix datasource error * fix jdbc error --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

* fix datasource error * fix jdbc error * fix et-monitor error --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

* fix datasource error * fix jdbc error * fix et-monitor error * fix et-monitor error --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

…nkis into dev-1.9.0-webank

…-webank

…-webank # Conflicts: # linkis-computation-governance/linkis-client/linkis-cli/src/main/java/org/apache/linkis/cli/application/interactor/job/common/LogRetriever.java # linkis-computation-governance/linkis-client/linkis-cli/src/main/java/org/apache/linkis/cli/application/interactor/job/interactive/InteractiveJob.java # linkis-computation-governance/linkis-engineconn/linkis-computation-engineconn/src/main/scala/org/apache/linkis/engineconn/computation/executor/hook/PythonModuleLoadEngineConnHook.scala # linkis-computation-governance/linkis-engineconn/linkis-computation-engineconn/src/main/scala/org/apache/linkis/engineconn/computation/executor/service/TaskExecutionServiceImpl.scala # linkis-engineconn-plugins/hive/src/main/scala/org/apache/linkis/engineplugin/hive/executor/HiveEngineConnExecutor.scala # linkis-engineconn-plugins/spark/src/main/scala/org/apache/linkis/engineplugin/spark/factory/SparkEngineConnFactory.scala # linkis-extensions/linkis-et-monitor/src/main/java/org/apache/linkis/monitor/scheduled/UserDepartmentInfoSync.java # linkis-extensions/linkis-et-monitor/src/main/java/org/apache/linkis/monitor/until/HttpsUntils.java # linkis-extensions/linkis-et-monitor/src/main/scala/org/apache/linkis/monitor/jobhistory/jobtime/StarrocksTimeExceedRule.scala # linkis-public-enhancements/linkis-configuration/src/main/java/org/apache/linkis/configuration/restful/api/ConfigurationRestfulApi.java # linkis-public-enhancements/linkis-datasource/linkis-datasource-manager/server/src/main/java/org/apache/linkis/datasourcemanager/core/restful/DataSourceOperateRestfulApi.java # linkis-public-enhancements/linkis-udf-service/src/main/java/org/apache/linkis/udf/api/UDFRestfulApi.java # linkis-public-enhancements/linkis-udf-service/src/main/scala/org/apache/linkis/udf/api/rpc/UdfReceiver.scala

peacewong and others added 30 commits August 19, 2024 10:50

bak

6101f8a

修复json4s问题

adf9b5d

Merge remote-tracking branch 'origin/dev-1.7.0-webank' into dev-1.7.0…

6b48401

…-webank-merge

解决编译问题

8b45350

Dev 1.7.1 add python module load hook (#584)

934b205

* ai generate code init * 人工修改代码 * 人工修改提交

【1.7.1】python物料管理 (#583)

227511f

* 代码由AI自动生成接口和AI插件生成 * 代码由AI自动生成接口和AI插件生成 * 代码由人工修改 --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

人工修改 code format and add rpc func

01ea074

fix bug

3a513b2

Dev 1.7.0 python udf manager (#586)

7bc3a82

* 代码由AI自动生成接口和AI插件生成 * 代码由AI自动生成接口和AI插件生成 * 代码由人工修改 * bug fix * bug fix --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

fix bug

694935f

fix bug

60f743b

code format

7237bea

使用ai生成单元测试案例

97f5301

人工修改单元测试

f0fba9e

人工修改单元测试

f070f29

Dev 1.7.1 webank test (#587)

69c3091

* 使用ai生成单元测试案例 * 人工修改单元测试 * 人工修改单元测试 * add instance info

add log

51e8b70

computation 模块的代码合并，除Entrance都已经进行合并

554013a

Merge remote-tracking branch 'origin/dev-1.7.1-webank' into dev-1.7.0…

88f28f6

…-webank-merge # Conflicts: # linkis-computation-governance/linkis-manager/linkis-application-manager/src/main/scala/org/apache/linkis/manager/am/conf/AMConfiguration.scala

移除Json4s

eaab3b9

合并storage和hdfs模块

8532fc6

Fix Fs问题

cd4f586

Fix jdbc 复杂类型的值转换bug

43a9c7e

合并spring和orchestrator等模块

d8d3827

YYYYMMddHH 变量时分秒修改为取当前值

660266e

完成所有模块的合并

9a599d0

完成所有模块的合并

4a6b417

修复编译问题

82474e8

修复编译问题

9ad7048

修复编译问题

aa544f1

peacewong and others added 28 commits October 18, 2024 17:55

Merge remote-tracking branch 'origin/dev-1.9.0-webank' into dev-1.9.0…

6079ad6

…-webank

Hive 支持字符串带制表符号

b372c9c

修复编译问题

002dad1

修复客户端拉日志空指针问题

8a41397

需要加上execID的确认保存

4cac70d

[1.9.0]Added StarRocks task timeout alarm and timeout kill function (#…

f7b22a6

…615) * Added StarRocks task timeout alarm and timeout kill function * Added StarRocks task timeout alarm and timeout kill function --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

Add synchronized department user table scheduled tasks (#593)

f39700f

Co-authored-by: “v_kkhuang” <“[email protected]”>

change to yarn proxy url

4605855

fix compile error

3eaca16

修复重复问题

af9bf53

Merge remote-tracking branch 'origin/dev-1.9.0-webank' into dev-1.9.0…

3565322

…-webank

[1.9.0]Fix Hive dependency issues (#618)

b45a4c7

* fix hive engine error * Security work order repair * sync ddl --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

Fix get tables only get table bug

4eef5d4

Merge remote-tracking branch 'origin/dev-1.9.0-webank' into dev-1.9.0…

3f31d5f

…-webank

fix datasource error (#620)

c934f0b

Co-authored-by: “v_kkhuang” <“[email protected]”>

Python module optimization upload (#619)

70c2cab

Co-authored-by: “v_kkhuang” <“[email protected]”>

Dev 1.9.0 fix bug (#621)

5c8b21d

* fix datasource error * fix jdbc error --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

Dev 1.9.0 fix bug (#622)

dbaa832

* fix datasource error * fix jdbc error * fix et-monitor error --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

code format

2e960bb

Dev 1.9.0 fix bug (#623)

d9e94c1

* fix datasource error * fix jdbc error * fix et-monitor error * fix et-monitor error --------- Co-authored-by: “v_kkhuang” <“[email protected]”>

add udf func check

98195d2

Merge branch 'dev-1.9.0-webank' of https://github.com/WeDataSphere/li…

8272ced

…nkis into dev-1.9.0-webank

add udf func check

181a2e9

Merge remote-tracking branch 'origin/dev-1.9.0-webank' into dev-1.9.0…

98f7431

…-webank

v-kkhuang closed this Oct 29, 2024

v-kkhuang deleted the dev-1.9.0-code-fix branch October 29, 2024 08:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev 1.9.0 code fix #627

Dev 1.9.0 code fix #627

v-kkhuang commented Oct 29, 2024

Dev 1.9.0 code fix #627

Dev 1.9.0 code fix #627

Conversation

v-kkhuang commented Oct 29, 2024

What is the purpose of the change

Related issues/PRs

Brief change log

Checklist