Design for Same-Table-JOINs #1623

acarbonetto · 2023-05-10T16:35:13Z

Description

Added documentation for a same-table join proposal.

related: https://opensearch.org/docs/latest/field-types/join/

Issues Resolved

[FEATURE] Support for subqueries or chaining of queries #1441

Check List

New functionality includes testing.
- All tests pass, including unit test, integration test and doctest
New functionality has been documented.
- New functionality has javadoc added
- New functionality has user manual doc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

docs/dev/opensearch-same-table-join-query.md

* Add opensearch-same-table-join-query.md Signed-off-by: Andrew Carbonetto <[email protected]> * Update opensearch-same-table-join-query.md for first-pass review Signed-off-by: Andrew Carbonetto <[email protected]> * Add same-table-joins ER relations Signed-off-by: Andrew Carbonetto <[email protected]> * Update document for review comments Signed-off-by: Andrew Carbonetto <[email protected]> --------- Signed-off-by: Andrew Carbonetto <[email protected]>

Signed-off-by: Andrew Carbonetto <[email protected]>

codecov · 2023-06-06T21:54:39Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.27%. Comparing base (691012d) to head (886e076).
Report is 362 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #1623   +/-   ##
=========================================
  Coverage     97.27%   97.27%           
  Complexity     4330     4330           
=========================================
  Files           388      388           
  Lines         10807    10807           
  Branches        761      761           
=========================================
  Hits          10513    10513           
  Misses          287      287           
  Partials          7        7

Flag	Coverage Δ
sql-engine	`97.27% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Yury-Fridlyand · 2023-06-14T01:00:37Z

docs/dev/opensearch-same-table-join-query.md

+
+## 3. Assumptions/Dependencies
+
+* _shard must be defined for any query using a same-table join


Yury-Fridlyand · 2023-06-14T01:01:27Z

docs/dev/opensearch-same-table-join-query.md

+
+* _shard must be defined for any query using a same-table join
+* Same-Table joins requires that the following work is complete:
+    * Table/Indices can be aliased, for example: `SELECT t.fieldA, t.fieldB FROM table t WHERE t.fieldA = t.fieldB`


Maybe give an example with join?

Yury-Fridlyand · 2023-06-14T01:02:18Z

docs/dev/opensearch-same-table-join-query.md

+
+### 3.3. Storage Type
+
+* `reference to opensearch doc`


Yury-Fridlyand · 2023-06-14T01:07:10Z

docs/dev/opensearch-same-table-join-query.md

+
+Summary:
+
+Example query (SQL):


Share the sample data to make example easier to understand. Are you using dataset from IT? In that case it is enough to refer a file in IT and rename index name in the query.

Or you can add this the sample data to the PR.

I found it on the very bottom. Please add a link here to the data.

Yury-Fridlyand · 2023-06-14T01:10:19Z

docs/dev/opensearch-same-table-join-query.md

+
+### 4.3 Case SELECT... FROM... JOIN ON... WHERE parent relation
+
+Summary: In this case, we are joining from `child` to `parent` using the `has_parent` relation query. 


You are doing the same in 4.1, aren't you?

Yury-Fridlyand · 2023-06-14T01:26:01Z

docs/dev/opensearch-same-table-join-query.md

+
+### 6.1 Parser Syntax
+
+```antlrv4


Suggested change

```antlrv4

```antlr

Yury-Fridlyand · 2023-06-14T01:27:19Z

docs/dev/opensearch-same-table-join-query.md

+### 6.1 Parser Syntax
+
+```antlrv4
+fromClause


Maybe mark by comments to identify new/changed and old syntax rules or show it as a diff

Yury-Fridlyand · 2023-06-14T02:34:52Z

docs/dev/opensearch-same-table-join-query.md

+    Relation->>DataSourceService: getTable
+    DataSourceService-->>Relation: Table
+    Relation->>Table: getFieldTypes
+    Table->>TypeEnvironment: define


You are missing returns from TypeEnvironment and Table back to Relation

Yury-Fridlyand · 2023-06-14T02:35:22Z

docs/dev/opensearch-same-table-join-query.md

+    JoinRelation->>TypeEnvironment: getOpenSearchJoinType
+    TypeEnvironment->>JoinRelation: OpenSearchJoinType
+    JoinRelation->>JoinRelation: setJoinRelationContext
+    JoinRelation->>Analyzer: <<LogicalRelation>>


Suggested change

JoinRelation->>Analyzer: <<LogicalRelation>>

JoinRelation->>Analyzer: LogicalRelation

Yury-Fridlyand · 2023-06-14T02:38:28Z

docs/dev/opensearch-same-table-join-query.md

+
+## Annex B - Data for examples
+
+The following section includes data that has been formatted to be used by the OpenSearch `got/_bulk` ingest endpoint.


Mention that all data to be added by single index named got (not multiple indices)

Swiddis · 2024-12-27T18:42:25Z

Is this still in progress? Should we close it?

Swiddis · 2024-12-27T19:02:21Z

Closing as stale -- feel free to reopen if work is resumed

acarbonetto requested review from pjfitzgibbons, ps48, kavithacm, derek-ho, joshuali925, dai-chen, YANG-DB, rupal-bq, mengweieric, vmmusings, Swiddis, penghuo, seankao-az, MaxKsyunz and Yury-Fridlyand as code owners May 10, 2023 16:35

matthewryanwells reviewed May 11, 2023

View reviewed changes

docs/dev/opensearch-same-table-join-query.md Outdated Show resolved Hide resolved

acarbonetto force-pushed the integ-design-same-table-joins branch from 055c21b to f7feaab Compare June 6, 2023 21:52

acarbonetto requested review from anirudha, forestmvey and GumpacG as code owners June 6, 2023 21:52

Update same-table-join doc for comment

886e076

Signed-off-by: Andrew Carbonetto <[email protected]>

Yury-Fridlyand reviewed Jun 14, 2023

View reviewed changes

This was referenced Jul 13, 2023

Support parent/child join in SQL #236

Closed

[RFC] Logical Plan Optimizer rework #1872

Open

acarbonetto marked this pull request as draft August 8, 2023 17:37

Swiddis closed this Dec 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Design for Same-Table-JOINs #1623

Design for Same-Table-JOINs #1623

acarbonetto commented May 10, 2023

codecov bot commented Jun 6, 2023 •

edited

Loading

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Yury-Fridlyand Jun 14, 2023

Swiddis commented Dec 27, 2024

Swiddis commented Dec 27, 2024


		## 3. Assumptions/Dependencies

		* _shard must be defined for any query using a same-table join


		### 4.3 Case SELECT... FROM... JOIN ON... WHERE parent relation

		Summary: In this case, we are joining from `child` to `parent` using the `has_parent` relation query.

	JoinRelation->>Analyzer: <<LogicalRelation>>
	JoinRelation->>Analyzer: LogicalRelation


		## Annex B - Data for examples

		The following section includes data that has been formatted to be used by the OpenSearch `got/_bulk` ingest endpoint.

Design for Same-Table-JOINs #1623

Design for Same-Table-JOINs #1623

Conversation

acarbonetto commented May 10, 2023

Description

Issues Resolved

Check List

codecov bot commented Jun 6, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Swiddis commented Dec 27, 2024

Swiddis commented Dec 27, 2024

codecov bot commented Jun 6, 2023 •

edited

Loading