struct subfield and list element type #1449

yliuuuu · 2024-04-27T00:51:47Z

Relevant Issues

[Closes/Related To] Issue #XXX

Description

Adding support for parsing Struct type with field declaration and List with Element type in DDL.
- Note that for operator that takes in type as parameter: IS, CAST, etc, we do not allow struct type with field declaration or list type with element type declaration at the moment.
Removing support for using NULL and MISSING as data type in DDL.

Other Information

Updated Unreleased Section in CHANGELOG: [YES/NO]
- < If NO, why? >
Any backward-incompatible changes? [YES/NO]
- < If YES, why? >
- < For this purpose, we define backward-incompatible changes as changes that—when consumed—can potentially result in
  errors for users that are using our public APIs or the entities that have public visibility in our code-base. >
Any new external dependencies? [YES/NO]
- < If YES, which ones and why? >
- < In addition, please also mention any other alternatives you've considered and the reason they've been discarded >
Do your changes comply with the Contributing Guidelines
and Code Style Guidelines? [YES/NO]

License Information

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

github-actions · 2024-04-27T00:58:37Z

Conformance comparison report-Cross Engine

	Base (legacy)	eval	+/-
% Passing	92.52%	90.70%	-1.82%
✅ Passing	5383	5278	-105
❌ Failing	435	541	106
🔶 Ignored	0	0	0
Total Tests	5818	5819	1
Number passing in both: 5071

Number failing in both: 228

Number passing in legacy engine but fail in eval engine: 313

Number failing in legacy engine but pass in eval engine: 207
⁉️ CONFORMANCE REPORT REGRESSION DETECTED ⁉️
The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact.
207 test(s) were failing in legacy but now pass in eval. Before merging, confirm they are intended to pass.
The complete list can be found in GitHub CI summary, either from Step Summary or in the Artifact.

Conformance comparison report-Cross Commit-LEGACY

	Base (`607c4c0`)	`b7609c2`	+/-
% Passing	92.51%	92.52%	0.02%
✅ Passing	5382	5383	1
❌ Failing	436	435	-1
🔶 Ignored	0	0	0
Total Tests	5818	5818	0
Number passing in both: 5382

Number failing in both: 435

Number passing in Base (607c4c0) but now fail: 0

Number failing in Base (607c4c0) but now pass: 1
The following test(s) were previously failing but now pass. Before merging, confirm they are intended to pass:

Click here to see

MYSQL_SELECT_29, compileOption: LEGACY

Conformance comparison report-Cross Commit-EVAL

	Base (`607c4c0`)	`b7609c2`	+/-
% Passing	90.67%	90.70%	0.03%
✅ Passing	5276	5278	2
❌ Failing	543	541	-2
🔶 Ignored	0	0	0
Total Tests	5819	5819	0
Number passing in both: 5276

Number failing in both: 541

Number passing in Base (607c4c0) but now fail: 1

Number failing in Base (607c4c0) but now pass: 3
⁉️ CONFORMANCE REPORT REGRESSION DETECTED ⁉️. The following test(s) were previously passing but now fail:

Click here to see

Example 6 — Value Coercion, compileOption: LEGACY

The following test(s) were previously failing but now pass. Before merging, confirm they are intended to pass:

Click here to see

equiv tupleunion with select list, compileOption: PERMISSIVE
equiv tupleunion with select list, compileOption: LEGACY
Example 6 — Value Coercion, compileOption: LEGACY

partiql-ast/src/main/resources/partiql_ast.ion

partiql-parser/src/main/kotlin/org/partiql/parser/internal/PartiQLParserDefault.kt

partiql-parser/src/test/kotlin/org/partiql/parser/internal/PartiQLParserDDLTests.kt

partiql-ast/src/main/resources/partiql_ast.ion

Co-authored-by: Alan Cai <[email protected]>

am357 · 2024-04-30T20:36:29Z

partiql-parser/src/main/antlr/PartiQL.g4

    | datatype=DOUBLE PRECISION                                                                                        # TypeAtomic
    | datatype=(CHARACTER|CHAR|FLOAT|VARCHAR) ( PAREN_LEFT arg0=LITERAL_INTEGER PAREN_RIGHT )?                         # TypeArgSingle
    | CHARACTER VARYING ( PAREN_LEFT arg0=LITERAL_INTEGER PAREN_RIGHT )?                                               # TypeVarChar
    | datatype=(DECIMAL|DEC|NUMERIC) ( PAREN_LEFT arg0=LITERAL_INTEGER ( COMMA arg1=LITERAL_INTEGER )? PAREN_RIGHT )?  # TypeArgDouble
    | datatype=(TIME|TIMESTAMP) ( PAREN_LEFT precision=LITERAL_INTEGER PAREN_RIGHT )? (WITH TIME ZONE)?                # TypeTimeZone
+    | datatype=(STRUCT|TUPLE) (ANGLE_LEFT structAttr ( COMMA structAttr )* ANGLE_RIGHT)                                # TypeStruct


Since this is going to the 1.0, do we want to flesh out the open/close and other struct constraints in the syntax now; if not, how can we avoid backward incompatibility when we introduce the syntax for those?

In my mental model, I think that using SQL's CREATE TABLE get us a bag of closed struct. For creating shapes that is unique to PartiQL, a bag of opened struct, a bag of integers, etc. we can consider vend out a PartiQL specific syntax (i.e., CREATE SHAPE ?) for those.

Just like other deviations from Hive DDL, we may have to support such use-cases within the same syntax; let's discuss further about this.

Adding the summary of our discussion in an example here for posterity:

CREATE TABLE [OPEN] Foo ( s STRUCT<a: INT, b:STRING> OPEN, )

partiql-ast/src/main/resources/partiql_ast.ion

partiql-parser/src/main/antlr/PartiQL.g4

partiql-parser/src/test/kotlin/org/partiql/parser/internal/PartiQLParserDDLTests.kt

partiql-parser/src/main/antlr/PartiQLTokens.g4

partiql-parser/src/main/kotlin/org/partiql/parser/internal/PartiQLParserDefault.kt

partiql-parser/src/test/kotlin/org/partiql/parser/internal/PartiQLParserDDLTests.kt

partiql-ast/src/main/resources/partiql_ast.ion

partiql-parser/src/main/kotlin/org/partiql/parser/internal/PartiQLParserDefault.kt

Co-authored-by: Alan Cai <[email protected]>

Delete NEQ Token

This reverts commit 23f6fee

* Revert "V1 ddl extended keyword (#1455)" This reverts commit 2879f3a * Revert "struct subfield and list element type (#1449)" This reverts commit 23f6fee * Revert "run apiDump (#1447)" This reverts commit 607c4c0 * Revert "Support parsing for attribute and tuple level constraint (#1442)" This reverts commit abfc58d * fix post-revert build * submodule

struct subfield and list element type

85e3d46

additional tests

5c800a3

yliuuuu requested a review from alancai98 April 27, 2024 01:24

yliuuuu marked this pull request as ready for review April 29, 2024 18:04

yliuuuu requested a review from am357 April 29, 2024 22:25

alancai98 reviewed Apr 29, 2024

View reviewed changes

yliuuuu and others added 2 commits April 29, 2024 16:11

Apply suggestions from code review

0c6bd1d

Co-authored-by: Alan Cai <[email protected]>

address feedback

d15e53e

yliuuuu requested a review from alancai98 April 30, 2024 01:32

am357 reviewed Apr 30, 2024

View reviewed changes

alancai98 reviewed Apr 30, 2024

View reviewed changes

yliuuuu added 3 commits April 30, 2024 17:58

fix ANTLR lexer

c3bc757

test and apidump

b8fbf43

address additional feedback

ed25286

yliuuuu requested a review from alancai98 May 1, 2024 17:06

alancai98 reviewed May 1, 2024

View reviewed changes

yliuuuu and others added 2 commits May 1, 2024 12:05

Apply suggestions from code review

c2e6f42

Co-authored-by: Alan Cai <[email protected]>

Update PartiQLTokens.g4

43629eb

Delete NEQ Token

alancai98 approved these changes May 1, 2024

View reviewed changes

am357 merged commit 23f6fee into v1 May 1, 2024
14 checks passed

alancai98 mentioned this pull request Jun 28, 2024

[v1] bag constructor allows for whitespace between angle brackets #1498

Closed

yliuuuu added a commit that referenced this pull request Jul 25, 2024

Revert "struct subfield and list element type (#1449)"

65443e0

This reverts commit 23f6fee

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

struct subfield and list element type #1449

struct subfield and list element type #1449

yliuuuu commented Apr 27, 2024

github-actions bot commented Apr 27, 2024 •

edited

Loading

am357 Apr 30, 2024

yliuuuu May 1, 2024

am357 May 1, 2024

am357 May 1, 2024

struct subfield and list element type #1449

struct subfield and list element type #1449

Conversation

yliuuuu commented Apr 27, 2024

Relevant Issues

Description

Other Information

License Information

github-actions bot commented Apr 27, 2024 • edited Loading

Conformance comparison report-Cross Engine

Conformance comparison report-Cross Commit-LEGACY

Conformance comparison report-Cross Commit-EVAL

am357 Apr 30, 2024

Choose a reason for hiding this comment

yliuuuu May 1, 2024

Choose a reason for hiding this comment

am357 May 1, 2024

Choose a reason for hiding this comment

am357 May 1, 2024

Choose a reason for hiding this comment

github-actions bot commented Apr 27, 2024 •

edited

Loading