Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature](update) Support update on current_timestamp #25884

Merged
merged 12 commits into from
Nov 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Column definition list:

Column definition:

`column_name column_type [KEY] [aggr_type] [NULL] [AUTO_INCREMENT] [default_value] [column_comment]`
`column_name column_type [KEY] [aggr_type] [NULL] [AUTO_INCREMENT] [default_value] [on update current_timestamp] [column_comment]`

* `column_type`

Expand Down Expand Up @@ -142,6 +142,10 @@ Column definition list:
dt DATETIME DEFAULT CURRENT_TIMESTAMP
```

* `on update current_timestamp`

To indicate that whether the value of this column should be updated to the current timestamp (`current_timestamp`) when there is an update on the row. The feature is only available on unique table with merge-on-write enabled. Columns with this feature enabled must declare a default value, and the default value must be `current_timestamp`. If the precision of the timestamp is declared here, the timestamp precision in the default value of the column must be the same as the precision declared here."

Example:

```
Expand All @@ -152,6 +156,7 @@ Column definition list:
v2 BITMAP BITMAP_UNION,
v3 HLL HLL_UNION,
v4 INT SUM NOT NULL DEFAULT "1" COMMENT "This is column v4"
dt datetime(6) default current_timestamp(6) on update current_timestamp(6)
```

#### index_definition_list
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ distribution_desc
* `column_definition`
列定义:

`column_name column_type [KEY] [aggr_type] [NULL] [AUTO_INCREMENT] [default_value] [column_comment]`
`column_name column_type [KEY] [aggr_type] [NULL] [AUTO_INCREMENT] [default_value] [on update current_timestamp] [column_comment]`
* `column_type`
列类型,支持以下类型:
```
Expand Down Expand Up @@ -129,6 +129,10 @@ distribution_desc
// 只用于DATETIME类型,导入数据缺失该值时系统将赋予当前时间
dt DATETIME DEFAULT CURRENT_TIMESTAMP
```
* `on update current_timestamp`
bobhan1 marked this conversation as resolved.
Show resolved Hide resolved

是否在该行有列更新时将该列的值更新为当前时间(`current_timestamp`)。该特性只能在开启了merge-on-write的unique表上使用,开启了这个特性的列必须声明默认值,且默认值必须为`current_timestamp`。如果此处声明了时间戳的精度,则该列默认值中的时间戳精度必须与该处的时间戳精度相同。


示例:

Expand All @@ -140,6 +144,7 @@ distribution_desc
v2 BITMAP BITMAP_UNION,
v3 HLL HLL_UNION,
v4 INT SUM NOT NULL DEFAULT "1" COMMENT "This is column v4"
dt datetime(6) default current_timestamp(6) on update current_timestamp(6)
```

#### index_definition_list
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -462,7 +462,8 @@ columnDef
: colName=identifier type=dataType
KEY? (aggType=aggTypeDef)? ((NOT NULL) | NULL)?
(DEFAULT (nullValue=NULL | INTEGER_VALUE | stringValue=STRING_LITERAL
| CURRENT_TIMESTAMP (LEFT_PAREN precision=number RIGHT_PAREN)?))?
| CURRENT_TIMESTAMP (LEFT_PAREN defaultValuePrecision=number RIGHT_PAREN)?))?
(ON UPDATE CURRENT_TIMESTAMP (LEFT_PAREN onUpdateValuePrecision=number RIGHT_PAREN)?)?
(COMMENT comment=STRING_LITERAL)?
;

Expand Down
36 changes: 33 additions & 3 deletions fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,12 @@ public class Column implements Writable, GsonPostProcessable {

private boolean isCompoundKey = false;

@SerializedName(value = "hasOnUpdateDefaultValue")
private boolean hasOnUpdateDefaultValue = false;

@SerializedName(value = "onUpdateDefaultValueExprDef")
private DefaultValueExprDef onUpdateDefaultValueExprDef;

public Column() {
this.name = "";
this.type = Type.NULL;
Expand Down Expand Up @@ -170,24 +176,33 @@ public Column(String name, Type type, boolean isKey, AggregateType aggregateType
public Column(String name, Type type, boolean isKey, AggregateType aggregateType, boolean isAllowNull,
String defaultValue, String comment) {
this(name, type, isKey, aggregateType, isAllowNull, false, defaultValue, comment, true, null,
COLUMN_UNIQUE_ID_INIT_VALUE, defaultValue);
COLUMN_UNIQUE_ID_INIT_VALUE, defaultValue, false, null);
}

public Column(String name, Type type, boolean isKey, AggregateType aggregateType, boolean isAllowNull,
String comment, boolean visible, int colUniqueId) {
this(name, type, isKey, aggregateType, isAllowNull, false, null, comment, visible, null, colUniqueId, null);
this(name, type, isKey, aggregateType, isAllowNull, false, null, comment, visible, null, colUniqueId, null,
false, null);
}

public Column(String name, Type type, boolean isKey, AggregateType aggregateType, boolean isAllowNull,
String defaultValue, String comment, boolean visible, DefaultValueExprDef defaultValueExprDef,
int colUniqueId, String realDefaultValue) {
this(name, type, isKey, aggregateType, isAllowNull, false, defaultValue, comment, visible, defaultValueExprDef,
colUniqueId, realDefaultValue);
colUniqueId, realDefaultValue, false, null);
}

public Column(String name, Type type, boolean isKey, AggregateType aggregateType, boolean isAllowNull,
boolean isAutoInc, String defaultValue, String comment, boolean visible,
DefaultValueExprDef defaultValueExprDef, int colUniqueId, String realDefaultValue) {
this(name, type, isKey, aggregateType, isAllowNull, isAutoInc, defaultValue, comment, visible,
defaultValueExprDef, colUniqueId, realDefaultValue, false, null);
}

public Column(String name, Type type, boolean isKey, AggregateType aggregateType, boolean isAllowNull,
boolean isAutoInc, String defaultValue, String comment, boolean visible,
DefaultValueExprDef defaultValueExprDef, int colUniqueId, String realDefaultValue,
boolean hasOnUpdateDefaultValue, DefaultValueExprDef onUpdateDefaultValueExprDef) {
this.name = name;
if (this.name == null) {
this.name = "";
Expand All @@ -212,6 +227,8 @@ public Column(String name, Type type, boolean isKey, AggregateType aggregateType
this.children = new ArrayList<>();
createChildrenColumn(this.type, this);
this.uniqueId = colUniqueId;
this.hasOnUpdateDefaultValue = hasOnUpdateDefaultValue;
this.onUpdateDefaultValueExprDef = onUpdateDefaultValueExprDef;

if (type.isAggStateType()) {
AggStateType aggState = (AggStateType) type;
Expand Down Expand Up @@ -244,6 +261,8 @@ public Column(Column column) {
this.uniqueId = column.getUniqueId();
this.defineExpr = column.getDefineExpr();
this.defineName = column.getDefineName();
this.hasOnUpdateDefaultValue = column.hasOnUpdateDefaultValue;
this.onUpdateDefaultValueExprDef = column.onUpdateDefaultValueExprDef;
}

public void createChildrenColumn(Type type, Column column) {
Expand Down Expand Up @@ -489,6 +508,14 @@ public int getOlapColumnIndexSize() {
}
}

public boolean hasOnUpdateDefaultValue() {
return hasOnUpdateDefaultValue;
}

public Expr getOnUpdateDefaultValueExpr() {
return onUpdateDefaultValueExprDef.getExpr(type);
}

public TColumn toThrift() {
TColumn tColumn = new TColumn();
tColumn.setColumnName(removeNamePrefix(this.name));
Expand Down Expand Up @@ -766,6 +793,9 @@ public String toSql(boolean isUniqueTable, boolean isCompatible) {
sb.append(" DEFAULT \"").append(defaultValue).append("\"");
}
}
if (hasOnUpdateDefaultValue) {
sb.append(" ON UPDATE ").append(defaultValue).append("");
}
if (StringUtils.isNotBlank(comment)) {
sb.append(" COMMENT '").append(getComment(true)).append("'");
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2166,6 +2166,16 @@ private void createOlapTable(Database db, CreateTableStmt stmt) throws UserExcep
}
olapTable.setEnableSingleReplicaCompaction(enableSingleReplicaCompaction);

// check `update on current_timestamp`
if (!enableUniqueKeyMergeOnWrite) {
for (Column column : baseSchema) {
if (column.hasOnUpdateDefaultValue()) {
throw new DdlException("'ON UPDATE CURRENT_TIMESTAMP' is only supportted"
+ " in unique table with merge-on-write enabled.");
}
}
}

// analyze bloom filter columns
Set<String> bfColumns = null;
double bfFpp = 0;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -378,31 +378,9 @@ public PlanFragment visitPhysicalOlapTableSink(PhysicalOlapTableSink<? extends P
HashSet<String> partialUpdateCols = new HashSet<>();
boolean isPartialUpdate = olapTableSink.isPartialUpdate();
if (isPartialUpdate) {
OlapTable olapTable = olapTableSink.getTargetTable();
if (!olapTable.getEnableUniqueKeyMergeOnWrite()) {
throw new AnalysisException("Partial update is only allowed in"
+ "unique table with merge-on-write enabled.");
}
for (Column col : olapTable.getFullSchema()) {
boolean exists = false;
for (Column insertCol : olapTableSink.getCols()) {
if (insertCol.getName() != null && insertCol.getName().equals(col.getName())) {
exists = true;
break;
}
}
if (col.isKey() && !exists) {
throw new AnalysisException("Partial update should include all key columns, missing: "
+ col.getName());
}
}
for (Column col : olapTableSink.getCols()) {
partialUpdateCols.add(col.getName());
}
if (olapTable.hasSequenceCol() && olapTable.getSequenceMapCol() != null
&& partialUpdateCols.contains(olapTable.getSequenceMapCol())) {
partialUpdateCols.add(Column.SEQUENCE_COL);
}
}
TupleDescriptor olapTuple = context.generateTupleDesc();
List<Column> targetTableColumns = olapTableSink.getTargetTable().getFullSchema();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2148,6 +2148,7 @@ public ColumnDefinition visitColumnDef(ColumnDefContext ctx) {
boolean isNotNull = ctx.NOT() != null;
String aggTypeString = ctx.aggType != null ? ctx.aggType.getText() : null;
Optional<DefaultValue> defaultValue = Optional.empty();
Optional<DefaultValue> onUpdateDefaultValue = Optional.empty();
if (ctx.DEFAULT() != null) {
if (ctx.INTEGER_VALUE() != null) {
defaultValue = Optional.of(new DefaultValue(ctx.INTEGER_VALUE().getText()));
Expand All @@ -2156,14 +2157,24 @@ public ColumnDefinition visitColumnDef(ColumnDefContext ctx) {
} else if (ctx.nullValue != null) {
defaultValue = Optional.of(DefaultValue.NULL_DEFAULT_VALUE);
} else if (ctx.CURRENT_TIMESTAMP() != null) {
if (ctx.precision == null) {
if (ctx.defaultValuePrecision == null) {
defaultValue = Optional.of(DefaultValue.CURRENT_TIMESTAMP_DEFAULT_VALUE);
} else {
defaultValue = Optional.of(DefaultValue
.currentTimeStampDefaultValueWithPrecision(Long.valueOf(ctx.precision.getText())));
.currentTimeStampDefaultValueWithPrecision(
Long.valueOf(ctx.defaultValuePrecision.getText())));
}
}
}
if (ctx.UPDATE() != null) {
if (ctx.onUpdateValuePrecision == null) {
onUpdateDefaultValue = Optional.of(DefaultValue.CURRENT_TIMESTAMP_DEFAULT_VALUE);
} else {
onUpdateDefaultValue = Optional.of(DefaultValue
.currentTimeStampDefaultValueWithPrecision(
Long.valueOf(ctx.onUpdateValuePrecision.getText())));
}
}
AggregateType aggType = null;
if (aggTypeString != null) {
try {
Expand All @@ -2174,7 +2185,8 @@ public ColumnDefinition visitColumnDef(ColumnDefContext ctx) {
}
}
String comment = ctx.comment != null ? ctx.comment.getText() : "";
return new ColumnDefinition(colName, colType, isKey, aggType, !isNotNull, defaultValue, comment);
return new ColumnDefinition(colName, colType, isKey, aggType, !isNotNull, defaultValue,
onUpdateDefaultValue, comment);
}

@Override
Expand Down
Loading