Export of internal ZetaSQL changes.

-- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Added the sqlbuilder implementation for functions with argument aliases. -- Change by ZetaSQL Team <[email protected]>: Stores argument alias information in resolved ast by using `generic_argument_list` rather than `argument_list` if the function signature has aliased arguments. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Add MakeRangeFromValidatedInputs, similar to MakeStructFromValidatedInputs for performance-sensitive cases -- Change by ZetaSQL Team <[email protected]>: Fix the generation_expression syntax, the current syntax requires () after AS. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Instead of relying on column id for order to create catalog columns to resolved columns map for the target table scan, ResolvePathExpressionAsTableScan now takes the map to be populated as a function argument. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by Christoph Dibak <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Multiway UNNEST test case refactoring -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Make Textmapper the default parser, except when the opt-out flag is set. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Fix create table with struct field that requires type coercion. -- Change by ZetaSQL Team <[email protected]>: Added the compliance test cases for EXCEPT (ALL|DISTINCT) for INNER CORRESPONDING BY. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Lift the ErrorMessageOptions() refactor to error_helpers.h and update callers to MaybeUpdateErrorFromPayload() -- Change by John Fremlin <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Remove unnecessary check in ConvertInternalErrorLocationAndAdjustErrorString() -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by Christoph Dibak <[email protected]>: Allow LIMIT and TABLESAMPLE in public group joins -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Mark `IeeeDivide()` and friends as `no_sanitize("float-divide-by-zero")`. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Introduce an opt-out flag for Textmapper, in preparation for making it the default primary parser. -- Change by ZetaSQL Team <[email protected]>: Remove dead code in AnonymizationRewriter. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Add a convenience overload to ConvertInternalErrorMessage() that accepts AnalyzerOptions. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Add compliance tests for IS [NOT] DISTINCT over RANGE types. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Extend cosine and euclidean distance functions to work with float32 arrays. -- Change by ZetaSQL Team <[email protected]>: Handle value table results more consistently. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Enable FEATURE_TEXTMAPPER_PARSER by default. Calling EnableMaximumFeatures() now sets TM as the parser. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Benchmark for EUCLIDEAN_DISTANCE implementation -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Heap allocate several larger objects in FunctionResolver::ResolveTemplatedSQLFunctionCall to reduce stack space consumption. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Modify JSON functions to allow constant expressions for JSONPaths. -- Change by ZetaSQL Team <[email protected]>: Directly set child values instead of merging. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Modify JSON functions to allow constant expressions for JSONPaths. -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: No public description -- Change by ZetaSQL Team <[email protected]>: Improve a log line in the type factory. (And 23 more changes) GitOrigin-RevId: d5b7efe5a5d7157472b259c9c595c5d2ebb6f410 Change-Id: I68e3d0af8010af8adef7082902b0e2b80abb5322
google · Nov 10, 2023 · 589026c · 589026c
1 parent a745bef
commit 589026c
Show file tree

Hide file tree

Showing 274 changed files with 29,864 additions and 3,737 deletions.
diff --git a/bazel/zetasql_deps_step_2.bzl b/bazel/zetasql_deps_step_2.bzl
@@ -117,9 +117,9 @@ def zetasql_deps_step_2(
         if not native.existing_rule("six_archive"):
             http_archive(
                 name = "six_archive",
-                url = "https://pypi.python.org/packages/source/s/six/six-1.10.0.tar.gz",
-                sha256 = "105f8d68616f8248e24bf0e9372ef04d3cc10104f1980f54d57b2ce73a5ad56a",
-                strip_prefix = "six-1.10.0",
+                url = "https://pypi.python.org/packages/source/s/six/six-1.16.0.tar.gz",
+                sha256 = "1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926",
+                strip_prefix = "six-1.16.0",
                 build_file_content = """licenses(["notice"])
 
 exports_files(["LICENSE"])

diff --git a/docs/aggregate_functions.md b/docs/aggregate_functions.md
@@ -101,6 +101,16 @@ To learn about the syntax for aggregate function calls, see
   </td>
 </tr>
 
+<tr>
+  <td><a href="#grouping"><code>GROUPING</code></a>
+
+</td>
+  <td>
+    Checks if a groupable value in the <code>GROUP BY</code> clause is
+    aggregated.
+  </td>
+</tr>
+
 <tr>
   <td><a href="#logical_and"><code>LOGICAL_AND</code></a>
 
@@ -1124,6 +1134,127 @@ FROM UNNEST([5, -2, 3, 6, -10, NULL, -7, 4, 0]) AS x;
  *------+--------------*/
 ```
 
+### `GROUPING`
+
+```sql
+GROUPING(groupable_value)
+```
+
+**Description**
+
+If a groupable item in the [`GROUP BY` clause][group-by-clause] is aggregated
+(and thus not grouped), this function returns `1`. Otherwise,
+this function returns `0`.
+
+Definitions:
+
++ `groupable_value`: An expression that represents a value that can be grouped
+  in the `GROUP BY` clause.
+
+Details:
+
+The `GROUPING` function is helpful if you need to determine which rows are
+produced by which grouping sets. A grouping set is a group of columns by which
+rows can be grouped together. So, if you need to filter rows by
+a few specific grouping sets, you can use the `GROUPING` function to identify
+which grouping sets grouped which rows by creating a matrix of the results.
+
+In addition, you can use the `GROUPING` function to determine the type of
+`NULL` produced by the `GROUP BY` clause. In some cases, the `GROUP BY` clause
+produces a `NULL` placeholder. This placeholder represents all groupable items
+that are aggregated (not grouped) in the current grouping set. This is different
+from a standard `NULL`, which can also be produced by a query.
+
+For more information, see the following examples.
+
+**Returned Data Type**
+
+`INT64`
+
+**Examples**
+
+In the following example, it's difficult to determine which rows are grouped by
+the grouping value `product_type` or `product_name`. The `GROUPING` function
+makes this easier to determine.
+
+Pay close attention to what's in the `product_type_agg` and
+`product_name_agg` column matrix. This determines how the rows are grouped.
+
+`product_type_agg` | `product_name_agg` | Notes
+------------------ | -------------------| ------
+1                  | 0                  | Rows are grouped by `product_name`.
+0                  | 1                  | Rows are grouped by `product_type`.
+0                  | 0                  | Rows are grouped by `product_type` and `product_name`.
+1                  | 1                  | Grand total row.
+
+```sql
+WITH
+  Products AS (
+    SELECT 'shirt' AS product_type, 't-shirt' AS product_name, 3 AS product_count UNION ALL
+    SELECT 'shirt', 't-shirt', 8 UNION ALL
+    SELECT 'shirt', 'polo', 25 UNION ALL
+    SELECT 'pants', 'jeans', 6
+  )
+SELECT
+  product_type,
+  product_name,
+  SUM(product_count) AS product_sum,
+  GROUPING(product_type) AS product_type_agg,
+  GROUPING(product_name) AS product_name_agg,
+FROM Products
+GROUP BY GROUPING SETS(product_type, product_name, ())
+ORDER BY product_name;
+
+/*--------------+--------------+-------------+------------------+------------------+
+ | product_type | product_name | product_sum | product_type_agg | product_name_agg |
+ +--------------+--------------+-------------+------------------+------------------+
+ | NULL         | NULL         | 36          | 1                | 1                |
+ | shirt        | NULL         | 36          | 0                | 1                |
+ | pants        | NULL         | 6           | 0                | 1                |
+ | NULL         | jeans        | 6           | 1                | 0                |
+ | NULL         | polo         | 25          | 1                | 0                |
+ | NULL         | t-shirt      | 11          | 1                | 0                |
+ +--------------+--------------+-------------+------------------+------------------*/
+```
+
+In the following example, it's difficult to determine
+if `NULL` represents a `NULL` placeholder or a standard `NULL` value in the
+`product_type` column. The `GROUPING` function makes it easier to
+determine what type of `NULL` is being produced. If
+`product_type_is_aggregated` is `1`, the `NULL` value for
+the `product_type` column is a `NULL` placeholder.
+
+```sql
+WITH
+  Products AS (
+    SELECT 'shirt' AS product_type, 't-shirt' AS product_name, 3 AS product_count UNION ALL
+    SELECT 'shirt', 't-shirt', 8 UNION ALL
+    SELECT NULL, 'polo', 25 UNION ALL
+    SELECT 'pants', 'jeans', 6
+  )
+SELECT
+  product_type,
+  product_name,
+  SUM(product_count) AS product_sum,
+  GROUPING(product_type) AS product_type_is_aggregated
+FROM Products
+GROUP BY GROUPING SETS(product_type, product_name)
+ORDER BY product_name;
+
+/*--------------+--------------+-------------+----------------------------+
+ | product_type | product_name | product_sum | product_type_is_aggregated |
+ +--------------+--------------+-------------+----------------------------+
+ | shirt        | NULL         | 36          | 0                          |
+ | NULL         | NULL         | 25          | 0                          |
+ | pants        | NULL         | 6           | 0                          |
+ | NULL         | jeans        | 6           | 1                          |
+ | NULL         | polo         | 25          | 1                          |
+ | NULL         | t-shirt      | 11          | 1                          |
+ +--------------+--------------+-------------+----------------------------*/
+```
+
+[group-by-clause]: https://github.com/google/zetasql/blob/master/docs/query-syntax.md#group_by_clause
+
 ### `LOGICAL_AND`
 
 ```sql