Skip to content

Commit

Permalink
bugfix : check if table meta cache should be refreshed in AT mode (ap…
Browse files Browse the repository at this point in the history
…ache#4734)

(cherry picked from commit 28b3413)
  • Loading branch information
Bughue committed Feb 28, 2023
1 parent 5aa6c29 commit 0f1f8af
Show file tree
Hide file tree
Showing 10 changed files with 280 additions and 45 deletions.
29 changes: 28 additions & 1 deletion changes/en-us/develop.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,41 @@ Add changes here for all PR submitted to the develop branch.
- [[#5023](https://github.com/seata/seata/pull/5203)] Fix `seata-core` dependency transitive conflict in `seata-dubbo`
- [[#5224](https://github.com/seata/seata/pull/5224)] fix oracle initialize script index_name is duplicate
- [[#5233](https://github.com/seata/seata/pull/5233)] fix the inconsistent configuration item names related to LoadBalance
- [[#5266](https://github.com/seata/seata/pull/5265)] fix server console has queried the released lock
- [[#5245](https://github.com/seata/seata/pull/5245)] fix the incomplete dependency of distribution module
- [[#5239](https://github.com/seata/seata/pull/5239)] fix `getConfig` throw `ClassCastException` when use JDK proxy
- [[#5281](https://github.com/seata/seata/pull/5281)] parallel request handle throw IndexOutOfBoundsException
- [[#5288](https://github.com/seata/seata/pull/5288)] fix auto-increment of pk columns in Oracle in AT mode
- [[#5287](https://github.com/seata/seata/pull/5287)] fix auto-increment of pk columns in PostgreSQL in AT mode
- [[#5299](https://github.com/seata/seata/pull/5299)] fix GlobalSession deletion when retry rollback or retry commit timeout
- [[#5307](https://github.com/seata/seata/pull/5307)] fix that keywords don't add escaped characters
- [[#5311](https://github.com/seata/seata/pull/5311)] remove RollbackRetryTimeout sessions during in file storage recover
- [[#4734](https://github.com/seata/seata/pull/4734)] check if table meta cache should be refreshed in AT mode
- [[#5316](https://github.com/seata/seata/pull/5316)] fix G1 jvm parameter in jdk8
- [[#5321](https://github.com/seata/seata/pull/5321)] fix When the rollback logic on the TC side returns RollbackFailed, the custom FailureHandler is not executed
- [[#5332](https://github.com/seata/seata/pull/5332)] fix bugs found in unit tests


### optimize:
- [[#5208](https://github.com/seata/seata/pull/5208)] optimize throwable getCause once more
- [[#5212](https://github.com/seata/seata/pull/5212)] optimize log message level
- [[#5237](https://github.com/seata/seata/pull/5237)] optimize exception log message print(EnhancedServiceLoader.loadFile#cahtch)
- [[#5089](https://github.com/seata/seata/pull/5089)] optimize the check of the delay value of the TCC fence log clean task
- [[#5243](https://github.com/seata/seata/pull/5243)] optimize kryo 5.4.0 optimize compatibility with jdk17
- [[#5153](https://github.com/seata/seata/pull/5153)] Only AT mode try to get channel with other app
- [[#5177](https://github.com/seata/seata/pull/5177)] If `server.session.enable-branch-async-remove` is true, delete the branch asynchronously and unlock it synchronously.
- [[#5273](https://github.com/seata/seata/pull/5273)] Optimize the compilation configuration of the `protobuf-maven-plugin` plug-in to solve the problem of too long command lines in higher versions.
- [[#5303](https://github.com/seata/seata/pull/5303)] remove startup script the -Xmn configuration
- [[#5325](https://github.com/seata/seata/pull/5325)] add store mode,config type and registry type log info
- [[#5315](https://github.com/seata/seata/pull/5315)] optimize the log of SPI
- [[#5323](https://github.com/seata/seata/pull/5323)] add time info for global transaction timeout log

### security:
- [[#5172](https://github.com/seata/seata/pull/5172)] fix some security vulnerabilities

### test:
- [[#xxx](https://github.com/seata/seata/pull/xxx)] add test for xxx
- [[#5380](https://github.com/seata/seata/pull/5380)] fix UpdateExecutorTest failed
- [[#5382](https://github.com/seata/seata/pull/5382)] fix multi spring version test failed

Thanks to these contributors for their code commits. Please report an unintended omission.

Expand All @@ -36,11 +55,19 @@ Thanks to these contributors for their code commits. Please report an unintended
- [albumenj](https://github.com/albumenj)
- [PeppaO](https://github.com/PeppaO)
- [yuruixin](https://github.com/yuruixin)
- [dmego](https://github.com/dmego)
- [CrazyLionLi](https://github.com/JavaLionLi)
- [xingfudeshi](https://github.com/xingfudeshi)
- [Bughue](https://github.com/Bughue)
- [pengten](https://github.com/pengten)
- [wangliang181230](https://github.com/wangliang181230)
- [GoodBoyCoder](https://github.com/GoodBoyCoder)
- [a364176773](https://github.com/a364176773)
- [isharpever](https://github.com/isharpever)
- [ZhangShiYeChina](https://github.com/ZhangShiYeChina)
- [mxsm](https://github.com/mxsm)
- [l81893521](https://github.com/l81893521)
- [liuqiufeng](https://github.com/liuqiufeng)


Also, we receive many valuable issues, questions and advices from our community. Thanks for you all.
30 changes: 29 additions & 1 deletion changes/zh-cn/develop.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,42 @@
- [[#5023](https://github.com/seata/seata/pull/5203)] 修复 `seata-core` 模块传递依赖冲突
- [[#5224](https://github.com/seata/seata/pull/5224)] 修复 oracle初始化脚本索引名重复的问题
- [[#5233](https://github.com/seata/seata/pull/5233)] 修复LoadBalance相关配置不一致的问题
- [[#5266](https://github.com/seata/seata/pull/5265)] 修复控制台全局锁查询接口查到了已释放的锁
- [[#5245](https://github.com/seata/seata/pull/5245)] 修复不完整的distribution模块依赖
- [[#5239](https://github.com/seata/seata/pull/5239)] 修复当使用JDK代理时,`getConfig` 方法获取部分配置时抛出 `ClassCastException` 异常的问题
- [[#5281](https://github.com/seata/seata/pull/5281)] 修复并行rm请求处理时数组索引越界问题
- [[#5288](https://github.com/seata/seata/pull/5288)] 修复AT模式下oracle的主键列自增的问题
- [[#5287](https://github.com/seata/seata/pull/5287)] 修复AT模式下pgsql的主键列自增的问题
- [[#5299](https://github.com/seata/seata/pull/5299)] 修复TC端重试回滚或重试提交超时GlobalSession的删除问题
- [[#5307](https://github.com/seata/seata/pull/5307)] 修复生成update前后镜像sql不对关键字转义的bug
- [[#5311](https://github.com/seata/seata/pull/5311)] 移除基于文件存储恢复时的RollbackRetryTimeout事务
- [[#4734](https://github.com/seata/seata/pull/4734)] 修复AT模式下新增字段产生的字段找不到
- [[#5316](https://github.com/seata/seata/pull/5316)] 修复jdk8 中 G1 参数
- [[#5321](https://github.com/seata/seata/pull/5321)] 修复当TC端回滚返回RollbackFailed时,自定义FailureHandler的方法未执行
- [[#5332](https://github.com/seata/seata/pull/5332)] 修复单元测试中发现的bug


### optimize:
- [[#5208](https://github.com/seata/seata/pull/5208)] 优化多次重复获取Throwable#getCause问题
- [[#5212](https://github.com/seata/seata/pull/5212)] 优化不合理的日志信息级别
- [[#5237](https://github.com/seata/seata/pull/5237)] 优化异常日志打印(EnhancedServiceLoader.loadFile#cahtch)
- [[#5089](https://github.com/seata/seata/pull/5089)] 优化 TCC fence log 清理定时任务的 delay 参数值检查
- [[#5243](https://github.com/seata/seata/pull/5243)] 升级 kryo 5.4.0 优化对jdk17的兼容性
- [[#5153](https://github.com/seata/seata/pull/5153)] 只允许AT去尝试跨RM获取channel
- [[#5177](https://github.com/seata/seata/pull/5177)] 如果 `server.session.enable-branch-async-remove` 为真,异步删除分支,同步解锁。
- [[#5273](https://github.com/seata/seata/pull/5273)] 优化`protobuf-maven-plugin`插件的编译配置,解决高版本的命令行过长问题
- [[#5303](https://github.com/seata/seata/pull/5303)] 移除启动脚本的-Xmn参数
- [[#5325](https://github.com/seata/seata/pull/5325)] 添加配置中心、注册中心类型以及存储模式日志信息
- [[#5315](https://github.com/seata/seata/pull/5315)] 优化SPI加载日志
- [[#5323](https://github.com/seata/seata/pull/5323)] 为全局事务超时日志添加时间信息


### security:
- [[#5172](https://github.com/seata/seata/pull/5172)] 修复一些安全漏洞的版本

### test:
- [[#xxx](https://github.com/seata/seata/pull/xxx)] 增加 xxx 测试
- [[#5380](https://github.com/seata/seata/pull/5380)] 修复 UpdateExecutorTest 单测失败问题
- [[#5382](https://github.com/seata/seata/pull/5382)] 修复多Spring版本测试失败

非常感谢以下 contributors 的代码贡献。若有无意遗漏,请报告。

Expand All @@ -36,11 +56,19 @@
- [albumenj](https://github.com/albumenj)
- [PeppaO](https://github.com/PeppaO)
- [yuruixin](https://github.com/yuruixin)
- [dmego](https://github.com/dmego)
- [CrazyLionLi](https://github.com/JavaLionLi)
- [xingfudeshi](https://github.com/xingfudeshi)
- [Bughue](https://github.com/Bughue)
- [pengten](https://github.com/pengten)
- [wangliang181230](https://github.com/wangliang181230)
- [GoodBoyCoder](https://github.com/GoodBoyCoder)
- [a364176773](https://github.com/a364176773)
- [isharpever](https://github.com/isharpever)
- [ZhangShiYeChina](https://github.com/ZhangShiYeChina)
- [mxsm](https://github.com/mxsm)
- [l81893521](https://github.com/l81893521)
- [liuqiufeng](https://github.com/liuqiufeng)


同时,我们收到了社区反馈的很多有价值的issue和建议,非常感谢大家。
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,13 @@
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.ScheduledThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

import javax.sql.DataSource;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import io.seata.common.Constants;
import io.seata.common.thread.NamedThreadFactory;
import io.seata.config.ConfigurationFactory;
import io.seata.common.ConfigurationKeys;
import io.seata.core.constants.DBType;
import io.seata.core.context.RootContext;
import io.seata.core.model.BranchType;
Expand All @@ -36,11 +34,6 @@
import io.seata.rm.datasource.sql.struct.TableMetaCacheFactory;
import io.seata.rm.datasource.util.JdbcUtils;
import io.seata.sqlparser.util.JdbcConstants;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import static io.seata.common.DefaultValues.DEFAULT_CLIENT_TABLE_META_CHECK_ENABLE;
import static io.seata.common.DefaultValues.DEFAULT_TABLE_META_CHECKER_INTERVAL;

/**
* The type Data source proxy.
Expand All @@ -65,21 +58,6 @@ public class DataSourceProxy extends AbstractDataSourceProxy implements Resource

private String version;

/**
* Enable the table meta checker
*/
private static boolean ENABLE_TABLE_META_CHECKER_ENABLE = ConfigurationFactory.getInstance().getBoolean(
ConfigurationKeys.CLIENT_TABLE_META_CHECK_ENABLE, DEFAULT_CLIENT_TABLE_META_CHECK_ENABLE);

/**
* Table meta checker interval
*/
private static final long TABLE_META_CHECKER_INTERVAL = ConfigurationFactory.getInstance().getLong(
ConfigurationKeys.CLIENT_TABLE_META_CHECKER_INTERVAL, DEFAULT_TABLE_META_CHECKER_INTERVAL);

private final ScheduledExecutorService tableMetaExecutor = new ScheduledThreadPoolExecutor(1,
new NamedThreadFactory("tableMetaChecker", 1, true));

/**
* Instantiates a new Data source proxy.
*
Expand Down Expand Up @@ -120,20 +98,18 @@ private void init(DataSource dataSource, String resourceGroupId) {
}
initResourceId();
DefaultResourceManager.get().registerResource(this);
if (ENABLE_TABLE_META_CHECKER_ENABLE) {
tableMetaExecutor.scheduleAtFixedRate(() -> {
try (Connection connection = dataSource.getConnection()) {
TableMetaCacheFactory.getTableMetaCache(DataSourceProxy.this.getDbType())
.refresh(connection, DataSourceProxy.this.getResourceId());
} catch (Exception ignore) {
}
}, 0, TABLE_META_CHECKER_INTERVAL, TimeUnit.MILLISECONDS);
}

TableMetaCacheFactory.registerTableMeta(this);
//Set the default branch type to 'AT' in the RootContext.
RootContext.setDefaultBranchType(this.getBranchType());
}

/**
* publish tableMeta refresh event
*/
public void tableMetaRefreshEvent() {
TableMetaCacheFactory.tableMetaRefreshEvent(this.getResourceId());
}

/**
* Gets plain connection.
*
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
* Copyright 1999-2019 Seata.io Group.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*/
package io.seata.rm.datasource.exception;

import java.sql.SQLException;

/**
* The type TableMetaException exception.
*
* @author Bughue
*/
public class TableMetaException extends SQLException {
private String columnName;
private String tableName;

public TableMetaException(String tableName, String columnName) {
this.columnName = columnName;
this.tableName = tableName;
}

public String getTableName() {
return tableName;
}

public String getColumnName() {
return columnName;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
import io.seata.rm.datasource.ConnectionContext;
import io.seata.rm.datasource.ConnectionProxy;
import io.seata.rm.datasource.StatementProxy;
import io.seata.rm.datasource.exception.TableMetaException;
import io.seata.rm.datasource.sql.struct.TableRecords;
import io.seata.sqlparser.SQLRecognizer;
import org.slf4j.Logger;
Expand Down Expand Up @@ -93,11 +94,18 @@ public T doExecute(Object... args) throws Throwable {
* @throws Exception the exception
*/
protected T executeAutoCommitFalse(Object[] args) throws Exception {
TableRecords beforeImage = beforeImage();
T result = statementCallback.execute(statementProxy.getTargetStatement(), args);
TableRecords afterImage = afterImage(beforeImage);
prepareUndoLog(beforeImage, afterImage);
return result;
try {
TableRecords beforeImage = beforeImage();
T result = statementCallback.execute(statementProxy.getTargetStatement(), args);
TableRecords afterImage = afterImage(beforeImage);
prepareUndoLog(beforeImage, afterImage);
return result;
} catch (TableMetaException e) {
LOGGER.error("table meta will be refreshed later, due to TableMetaException, table:{}, column:{}",
e.getTableName(), e.getColumnName());
statementProxy.getConnectionProxy().getDataSourceProxy().tableMetaRefreshEvent();
throw e;
}
}

private boolean isMultiPk() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,7 @@ protected TableMeta getTableMeta(String tableName) {
* @param afterImage the after image
* @return sql undo log
*/
@Override
protected SQLUndoLog buildUndoItem(TableRecords beforeImage, TableRecords afterImage) {
SQLType sqlType = sqlRecognizer.getSQLType();
String tableName = beforeImage.getTableName();
Expand Down
Loading

0 comments on commit 0f1f8af

Please sign in to comment.