Enhance the processing capacity of Linkis RPC, and maintain the backward compatibility with Linkis0.x RPC #614
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ard compatibility with Linkis0.x RPC closed #569
peacewong
added
component=Commons/MessageScheduler
mayinrain
pushed a commit
to mayinrain/incubator-linkis
that referenced
this pull request
Nov 19, 2024
* [1.9.0]Added StarRocks task timeout alarm and timeout kill function (apache#615) * Added StarRocks task timeout alarm and timeout kill function * Added StarRocks task timeout alarm and timeout kill function --------- Co-authored-by: “v_kkhuang” <“[email protected]”> * Improve the handling of decimal data types (apache#614) -Change DecimalType to class, supporting different precision and decimal places -Modify the dataType recognition logic to support various representations of the decimal type -Update Excel write logic to correctly handle different decimal types Co-authored-by: “v_kkhuang” <“[email protected]”> * [1.9.0]Data source password encryption enhancement (apache#612) * Data source password encryption enhancement Introduce AES encryption mechanism to enhance the security of data source passwords. This submission includes the implementation of encrypting and decrypting data source passwords to ensure password security during storage and transmission. * Using Error Exception to Handle Exceptions in AESUtils * 加密和密码管理改进 - 更新了AESUtils中的错误异常消息，以符合代码规范。 - 在ConfigurationRestfulApi中，对特定配置的密码进行了加密处理，以增强安全性。 - 修改了DataSourceCoreRestfulApi中的注释，并对密码字段的加密处理进行了优化。 - 在NebulaEngineConnExecutor中，密码现在使用AES进行解密，以确保数据安全性。 * Code optimization, adding data source encryption switch * Code optimization, adding data source encryption switch * Update AESUtils.java update default key --------- Co-authored-by: “v_kkhuang” <“[email protected]”> Co-authored-by: Casion <[email protected]> * Add synchronized department user table scheduled tasks (apache#593) Co-authored-by: “v_kkhuang” <“[email protected]”> * [1.9.0]Add department level result set restriction configuration and inspection (apache#617) * 添加部门级别的结果集限制配置和检查 - 在 Configuration.scala 中添加 JOB_RESULT_DEPARTMENT_LIMIT 变量，用于配置部门结果集限制- 实现 canResultSetByDepartment 方法，根据部门 ID 判断是否有结果集权限 - 在 QueryRestfulApi.java 中调用 canResultSetByDepartment 方法，将结果加入 API响应 * Update Configuration.scala update default department id --------- Co-authored-by: “v_kkhuang” <“[email protected]”> Co-authored-by: Casion <[email protected]> * [1.9.0]Fix Hive dependency issues (apache#618) * fix hive engine error * Security work order repair * sync ddl --------- Co-authored-by: “v_kkhuang” <“[email protected]”> * fix datasource error (apache#620) Co-authored-by: “v_kkhuang” <“[email protected]”> * Python module optimization upload (apache#619) Co-authored-by: “v_kkhuang” <“[email protected]”> * Dev 1.9.0 fix bug (apache#621) * fix datasource error * fix jdbc error --------- Co-authored-by: “v_kkhuang” <“[email protected]”> * Dev 1.9.0 fix bug (apache#622) * fix datasource error * fix jdbc error * fix et-monitor error --------- Co-authored-by: “v_kkhuang” <“[email protected]”> * Dev 1.9.0 fix bug (apache#623) * fix datasource error * fix jdbc error * fix et-monitor error * fix et-monitor error --------- Co-authored-by: “v_kkhuang” <“[email protected]”> * Code optimization * Code optimization --------- Co-authored-by: “v_kkhuang” <“[email protected]”> Co-authored-by: Casion <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
closed #569
Add the Linkis Message-Scheduler module to enhance the processing capacity of Linkis RPC.
Brief change log
Support synchronously and asynchronously processing message, and the receiver is identified through annotations.
When processing messages, support sending messages to other receivers in the meanwhile. For example, after EngineConn is started, AM needs to notify Label to register the label information of EC.
Messages are able to be processed by multiple receivers at the same time. For example, ECM registration request needs to register basic information in AM and register resource information in RM.
Support multiple receivers to execute in a single chain sequence, and multually transfer intermediate variables. For example, when EngineConn exits, you need to complete RM information logout, label information logout and AM instance information processing in order.
Support multiple receivers to execute in the order of multiple chains.
Support skipping receivers, for example, after a receiver has been completed and satisfied the conditions, you can skip the subsequent message receivers.
Support integrating RestFul with Message-scheduler, and thus reducing the development of the repeated logic between RPC and RestFul.