Release notes for 367

[martint]

Ashhar Hasan

• all checked
• 12-16 05:25:23 b294765 Don't annotate checkstyle and errorprone warnings
• 12-16 05:25:23 ca1e6f2 Use proper path glob when uploading product test reports
• 12-16 10:28:34 c41be97 Fix cleanup workflow not working for PRs from forks
• 12-17 11:02:53 e231b45 Document predicate pushdown in the Oracle connector

Dain Sundstrom

• all checked
• 12-16 19:37:36 14dc2d2 Clean up PagesIndex
• 12-16 19:37:36 1866a23 Remove argument channels from WindowFunctionSupplier
• 12-16 19:37:36 24b7827 Move Aggregator to aggregation package
• 12-16 19:37:36 408ca55 Make aggregation combine optional
• 12-16 19:37:36 5949565 Move aggregation update and merge to state interfaces
• 12-16 19:37:36 59c1c30 Move aggregation to window conversion to LocalExecutionPlanner
• 12-16 19:37:36 86393fd Remove parameter kind from aggregation implementation
• 12-16 19:37:36 991c089 Add raw block access to PagesWindowIndex for window aggregation
• 12-16 19:37:36 992c708 Simplify aggregation operator usage of functions
• 12-16 19:37:36 9c6a7e2 Simplify WindowFunctionSupplier interface
• 12-16 19:37:36 a1163b4 Move aggregation compilation to LocalExecutionPlanner
• 12-16 19:37:36 e3b66fa Simplify aggregation accumulator
• 12-16 19:37:36 e423abf Simplify AggregateWindowFunction
• 12-16 19:37:36 eedd97f Normalize aggregation input calling convention to block position
• 12-16 22:01:08 26cd967 Replace unsafe copy method with copy constructor
• 12-16 22:01:08 9745908 Ensure that state factory does not create a null state
• 12-19 15:47:14 3d5d23f Fix ClassNotFoundException when generating GroupedAccumulatorState
• 12-19 15:47:14 744ee5e Generated GroupedAccumulatorStatee shouldn't implement GroupedAccumulator
• 12-19 18:13:38 0e4022e Replace most uses of TypeRegistry with TypeManager
• 12-19 18:13:38 23b7d12 Remove TablePropertyManager from Metadata
• 12-19 18:13:38 2aa2339 Remove unused getAnalyzerFactory methods
• 12-19 18:13:38 309efbb Remove type registration from Metadata
• 12-19 18:13:38 403aa88 Add PlannerContext
• 12-19 18:13:38 4c514e1 Remove MaterializedViewPropertyManager from Metadata
• 12-19 18:13:38 723f6bd Simplify PlanOptimizersProvider interface in LocalQueryRunner
• 12-19 18:13:38 7977660 Remove AnalyzePropertyManager from Metadata
• 12-19 18:13:38 8e40a44 Replace Metadata type methods with TypeManager
• 12-19 18:13:38 8f269ec Remove SessionPropertyManager from Metadata
• 12-19 18:13:38 92822c7 Cleanup explain query testing infrastructure
• 12-19 18:13:38 b96ef25 Remove ColumnPropertyManager from Metadata
• 12-19 18:13:38 be9650c Remove BlockEncodingSerde from metadata
• 12-19 18:13:38 c13f952 Remove createExpressionAnalyzer from StatementAnalyzerFactory
• 12-19 18:13:38 d90e79d Remove TableProceduresPropertyManager from Metadata
• 12-19 18:13:38 e02d5c2 Simplify statement and expression analyzer creation
• 12-19 18:13:38 e37af15 Remove SchemaPropertyManager from Metadata
• 12-19 18:13:38 e7e1ac7 Move session to from a field to a parameter in LiteralEncoder
• 12-19 18:13:38 e988de1 Remove TableProceduresRegistry from Metadata
• 12-19 18:13:38 e9ee7d8 Simplify SetTimeZoneTask
• 12-19 18:13:38 ed31a48 Remove block encoding management from MetadataManager
• 12-19 18:13:38 eee57d5 Remove ProcedureRegistry from Metadata

Grzegorz Kokosiński

• all checked
• 12-16 04:06:58 e8bcfe2 Allow to hide inaccessible columns
• 12-16 04:55:12 e2c6cfa Introduce a simple logout page
• 12-16 06:56:52 e4aba45 Add more Kerberos debugging JVM flags in docs

Karol Sobczak

• all checked
• 12-16 04:40:11 a60184c Add high level comments for some scheduling components
• 12-17 01:47:39 504c9cf Support dynamic filtering with query retries
• 12-17 01:47:39 f5a60ba Move test helper methods to AbstractTestQueryFramework
• 12-17 05:28:57 0424588 Add additional order by columns for tpc-ds query 65
• 12-17 05:28:57 16a5579 Fix channel column for store sales in tpc-ds query 77
• 12-17 05:28:57 52a2f03 Decrease precision of results for tpc-ds query 39
• 12-17 05:28:57 882636a Add additional order by columns for tpc-ds query 73
• 12-17 05:28:57 8e453ac Add additional order by columns for tpc-ds query 71
• 12-17 05:28:57 bba5638 Add additional order by columns for tpc-ds query 44
• 12-17 05:28:57 d699c3f Add additional order by columns for tpc-h query 11
• 12-20 03:14:54 332c452 Add option to benchmark only selected methods

Martin Traverso

• all checked
• 12-16 18:22:19 909a910 Fix failure when selecting a constant and all fields from row
• 12-16 20:52:15 56949b3 Cache CanonicalizationAware hashcode
• 12-17 15:04:59 ef3e316 Fix failure handling in ExchangeClientBuffer
• 12-17 20:11:04 32de5c1 Fix noMoreBuffer signal for output buffers
• 12-17 20:11:04 f47463f Make number of splits configurable in TpchConnectorFactory
• 12-19 17:28:39 2cff28f Fix failure when row or array in VALUES contains nulls

Piotr Findeisen

• all checked
• 12-15 05:56:34 b8edc85 Increase test timeout
• 12-16 00:55:53 03cd814 Remove redundant check
• 12-16 00:55:53 3ac4e03 Reuse existing variable
• 12-16 00:55:53 798ae91 Avoid Optional.get without a check
• 12-16 00:55:53 8f1b333 Remove redundant null-friendliness
• 12-16 00:55:53 9b9f280 Unwrap from Optional that must bear a value
• 12-16 00:55:53 d4cc882 Rename argument for consistency
• 12-17 00:35:44 0a08ee5 Reuse request objects for AWS API pagination
• 12-17 00:35:44 5d8ff97 Add helper for reading all paginated results
• 12-17 00:35:44 851b6d1 Remove unused parameter
• 12-17 06:22:18 392b053 Remove skip_on_cdh test group
• 12-20 03:29:31 62ecf0d Add hive metastore cache flush procedure
• 12-20 03:29:31 e0ec56c Move hive procedures to dedicated package
• 12-20 03:40:34 453dcb2 Remove redundant else
• 12-20 03:40:34 561abfe Report query for query assertion failures
• 12-20 07:56:06 086b2f9 Pass schema type to FileHiveMetastore.getSchemaPath and its callers
• 12-20 07:56:06 1b73db6 Add deleteData parameter to HiveMetastore.dropDatabase
• 12-20 07:56:06 4dda775 Don't put database schema files in the database directories
• 12-20 07:56:06 6c2882f Rename database object-deletion method in FileHiveMetastore
• 12-20 07:56:06 719b0cf Delete database location after dropping in GlueHiveMetastore
• 12-20 07:56:06 a782eff Fix a typo
• 12-20 07:56:06 e8c5f7c Don't delete visibly non-empty directories when dropping Hive schemas
• 12-20 07:56:56 9a88595 Use Hive 3 Tempto config in Spark product test envs
• 12-20 07:56:56 ccee7b6 Only delete non-empty directories when dropping Iceberg schemas

Praveen Krishna

• all checked
• 12-20 01:37:38 087dc9f Capture column level dependencies for CTEs and sub-queries

Łukasz Osipiuk

• all checked
• 12-15 08:45:57 9366414 Disable distributed sort when failure recovery is enabled
• 12-15 08:52:15 7b0e9b2 Move query retry configs to QueryManagerConfig
• 12-20 05:52:35 70106a8 Remove duplicate variable
• 12-20 11:44:29 2a25a6b Add Hive failure recovery tests with bucketed tables
• 12-20 11:44:29 42d9eae Fix typo
• 12-20 11:44:29 46f08f8 Make retries not supported by connectors by default
• 12-20 11:44:29 5ed26e6 Add UUID to filenames for bucketed non-transactional tables
• 12-20 11:44:29 7593624 Run setup, cleanup and get expected results without retries
• 12-20 11:44:29 7a60926 Request path-style-access in S3HiveQueryRunner
• 12-20 11:44:29 7d9c3b1 Test timeout failure for all test cases
• 12-20 11:44:29 8291d33 Extract BaseTestHiveFailureRecovery
• 12-20 11:44:29 893ee55 Clean files for failed retry attempts for Hive
• 12-20 11:44:29 bf1e19a Extract computation of failure stage id out of executeActual
• 12-20 11:44:29 d23ff7a Decrease request timeouts in failure tests
• 12-20 11:44:29 d46c349 Run failure recovery tests once
• 12-20 11:44:29 e656049 Use new SPI methods with RetryMode in HiveMetadata
• 12-20 11:44:29 ee860dd Test Hive failure recovery with tables useign s3:// (via MinIO)
• 12-20 11:44:29 f8754c4 Check if queries fail without retries

[kokosing]

Security
* Allow to hide inaccessible columns from `SELECT *` statement when `hide-inaccessible-columns=true` configuration property is used. ({issue}`9991 `)

#9991

[kokosing]

Security/Web UI
* Fix logout from Web UI when Oauth2 authentication is used. ({issue}`10299`)

#10299

[sopel39]

General
* Improve retryable query performance by adding support for dynamic filtering. ({issue}`10274`) 

#10274

[findepi]

Hive, Iceberg
- Prevent data loss when schema location contains files, but not tables, during `DROP SCHEMA`. ({issue}`10146`, {issue}`9767`)

#9767 #10146

[losipiuk]

Hive
- Ensure no duplicate rows are created if query which writes data to Hive table is retried. ({issue}`10252`, {issue}`10064`)

#10064 #10252

[kokosing]

General
* Due security reasons, disable `SET AUTHORIZATION` when `VIEW` runs as `DEFINER`. ({issue}`10351`)

#10351

[aczajkowski]

Hive
 * introduce metadata cache flush procedure 

#10251

[aczajkowski]

General
 * introduce metadata cache flush procedure for JDBC connectors

#10265

[dain]

General
* Fix failure in `min_by` and `max_by` aggregation execution. ({issue}`10347`) 

SPI
* Fix `ClassNotFoundException` when  using aggregation with a custom state type.. ({issue}`10341`) 

#10347
#10341

[sopel39]

General:
* Improve `CREATE VIEW` error message when table or materialized view already exists. ({issue}`10186`)

#10186

[Praveen2112]

General:
* Capture lineage details for columns from Common table expressions and sub-queries. ({issue}`10272`)

#10319

[losipiuk]

General:
* Verify if it is safe to perform DML operation with query retries enabled. Now it will only be allowed for Hive connector with non-transactional tables. Otherwise DML query will be failed. ({issue}`10064`)

#10064

[martint]

• Verify if it is safe to perform DML operation with query retries enabled. Now it will only be allowed for Hive connector with non-transactional tables. Otherwise DML query will be failed. ({issue}10064)

@losipiuk, DML is not a "thing". Most users won't understand what that means. Would it be fair to say: "Disallow query retries when connectors cannot perform them safely"?