Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support map data type in yaml testing framework #3765

Merged
merged 1 commit into from
Mar 28, 2024

Conversation

aceforeverd
Copy link
Collaborator

allow map data type in YAML testing framework for input definition, you
may define input table's schema as:

columns: ["col1 int", "col2 map<int, string>"]

which is exactly the same as the SQL syntax in CREATE TABLE statement.

And define table values with inputs[*].inserts:

inserts:
- insert into t1 values (1, map(12, "abc"))

just write down the insert statement SQL.

LIMITATIONS

  • unsupported: map data type in expect field
  • INSERT statement with custom columns definition, implementation
    limits, each insert value expression must matches exactly to the table column definition

@github-actions github-actions bot added batch-engine openmldb batch(offline) engine execute-engine hybridse sql engine storage-engine openmldb storage engine. nameserver & tablet labels Feb 21, 2024
Copy link
Contributor

github-actions bot commented Feb 21, 2024

Linux Test Report

 53 files   60 suites   28m 28s ⏱️
671 tests 663 ✅ 7 💤 1 ❌
672 runs  664 ✅ 7 💤 1 ❌

For more details on these failures, see this check.

Results for commit 6121640.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Feb 21, 2024

HybridSE Mac Test Report

20 124 tests  +8   20 122 ✅ +8   9m 23s ⏱️ - 1m 24s
   256 suites ±0        2 💤 ±0 
    68 files   ±0        0 ❌ ±0 

Results for commit 6121640. ± Comparison against base commit aa803af.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Feb 21, 2024

HybridSE Linux Test Report

20 124 tests  +8   20 122 ✅ +8   6m 9s ⏱️ +6s
   256 suites ±0        2 💤 ±0 
    68 files   ±0        0 ❌ ±0 

Results for commit 6121640. ± Comparison against base commit aa803af.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Feb 21, 2024

SDK Test Report

101 files  ±0  101 suites  ±0   2m 9s ⏱️ +2s
349 tests ±0  335 ✅ ±0  14 💤 ±0  0 ❌ ±0 
475 runs  ±0  461 ✅ ±0  14 💤 ±0  0 ❌ ±0 

Results for commit 6121640. ± Comparison against base commit aa803af.

This pull request removes 30 and adds 9 tests. Note that renamed tests count towards both.
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
 ) limit 10;](2)
 ) limit 10;](3)
 FROM db1.t1
 FROM t1
 WINDOW w1 AS (
 last join db2.t2 order by db2.t2.col1
…
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[db1,  SELECT sum(t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[null,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](4)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db1.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: db1.t2.str1](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.col1](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.str1](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[null, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](5)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlWindowLastJoin[ SELECT sum(t1.col1) over w1 as sum_t1_col1, t2.str1 as t2_str1
 FROM t1
 last join t2 order by t2.col1
 on t1.col1 = t2.col1 and t1.col2 = t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)

♻️ This comment has been updated with latest results.

@aceforeverd aceforeverd force-pushed the feat-testing-map-type branch 3 times, most recently from 1c3a0c3 to 7a9c66b Compare February 21, 2024 09:35
Copy link

codecov bot commented Feb 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 40.77%. Comparing base (aa803af) to head (6121640).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #3765   +/-   ##
=========================================
  Coverage     40.77%   40.77%           
  Complexity      658      658           
=========================================
  Files           195      195           
  Lines         11659    11659           
  Branches       1379     1379           
=========================================
  Hits           4754     4754           
  Misses         6601     6601           
  Partials        304      304           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/sdk/sql_cluster_router.cc Outdated Show resolved Hide resolved
@dl239
Copy link
Collaborator

dl239 commented Mar 1, 2024

Is there performance comparison between codegen encoding and the original encoding?

src/cmd/sql_cmd_test.cc Outdated Show resolved Hide resolved
@aceforeverd
Copy link
Collaborator Author

Is there performance comparison between codegen encoding and the original encoding?

we should do some benchmark after codegen insert row generally stable (current alpha, interface may change).

Copy link
Collaborator

@tobegit3hub tobegit3hub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

allow map data type in YAML testing framework for input definition, you
may define input table's schema as:

  columns: ["col1 int", "col2 map<int, string>"]

which is exactly the same as the SQL syntax in CREATE TABLE statement.

And define table values with `inputs[*].inserts`:

  inserts:
    - insert into t1 values (1, map(12, "abc"))

just write down the insert statement SQL.

*LIMITATIONS*

- unsupported: map data type in `expect` field
- INSERT statement with custom columns definition, implementation
  limits, each insert value expression must matches exactly to the table column definition
@aceforeverd aceforeverd merged commit 86d640a into 4paradigm:main Mar 28, 2024
28 of 31 checks passed
@aceforeverd aceforeverd deleted the feat-testing-map-type branch March 28, 2024 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
execute-engine hybridse sql engine storage-engine openmldb storage engine. nameserver & tablet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants