Release notes for 360

[martint]

Ashhar Hasan

• all checked
• 07-06 10:56:19 70e849c Add insert_batch_size to JDBC metadata session properties
• 07-07 01:17:16 90c22f6 Remove redundant code
• 07-07 01:17:16 c7adc54 Replace toList with toUnmodifiableList() collector
• 07-07 05:40:03 903d940 Use a temporary table to verify if deletes are supported
• 07-07 05:40:03 baa7082 Simplify lambda to method reference
• 07-08 03:24:49 9fb91ca Use a temporary table to verify if deletes are supported
• 07-08 05:46:46 0761303 Improve naming in tests
• 07-08 22:37:13 f7f5639 Add list of supported statements for PostgreSQL connector
• 07-08 22:39:00 cc767dd Reword and extend list of supported SQL statements
• 07-09 03:49:21 ce496c5 Mark TestHiveStorageFormats#testCreateTableAs as flaky
• 07-09 10:24:44 d706dac Add requirements for Thrift connector
• 07-09 12:35:32 23b6ef4 Rename test method to clarify intent
• 07-09 12:35:32 4bbcb74 Migrate to skipTestUnless
• 07-09 12:35:32 4f6576e Fix declaration of SUPPORTS_DELETE in Cassandra tests
• 07-09 12:35:32 6198afb Remove redundant verification of delete support
• 07-09 12:35:32 8d122b2 Add tests to verify DELETE support declared by connectors
• 07-09 12:35:32 96be8f2 Use TestTable to ensure test tables are cleaned up
• 07-09 12:35:32 9bca437 Add smoke test for full table deletes
• 07-09 23:38:40 bbfca82 Don't exclude tests in trino-tests module to only run on CI
• 07-13 04:18:14 22ddd59 Allow bypassing temporary table during insert for jdbc-based connectors
• 07-14 12:04:30 640977d Simplify & clean up table handle lookup in DruidJdbcClient
• 07-14 23:39:37 e4ee1eb Fix Cassandra documentation for supported partition types
• 07-15 07:52:45 2c2b13f Revert "Simplify transactions in JdbcPageSink"
• 07-15 07:52:45 3a30d76 Enabled autocommit by default for pooled Oracle connections
• 07-15 07:52:45 543cd1f Document why disabling autocommit is important during writes
• 07-16 07:36:53 948e77a Add dummy tests to force testng respect single threaded
• 07-16 07:39:04 184bd50 Add QueryAssertions.matches(PlanMatchPattern) method.
• 07-16 07:39:04 fe3ba1b Allow Phoenix to replace partial topn with limit.
• 07-18 01:20:20 bf1f7f5 Remove lenient Number cast in TypeUtils#writeNativeValue
• 07-18 01:20:20 da30239 Change PartitionTable#convert to throw on overflow in multiplication
• 07-23 00:52:30 a71e032 Rename 'insert.batch-size' configuration property into 'write.batch-size'

Dain Sundstrom

• all checked
• 07-23 12:25:39 510caa8 Clarify and extend user mapping topic

David Phillips

• all checked
• 07-09 11:16:17 2300fbf Add list of supported statements for Iceberg connector
• 07-15 18:04:55 38f4e70 Remove unnecessary jackson-dataformat-cbor dependency
• 07-15 18:05:19 4492277 Minimize delays for high volume query ID generation
• 07-15 18:05:19 5752946 Convert QueryIdGenerator to java.time and fix warnings
• 07-15 18:05:19 961bf1e Simplify exception method in ExecutingStatementResource
• 07-15 18:05:19 e12534f Extract utility method to generate query info URI
• 07-16 13:51:22 b691db7 Add documentation for materialized views
• 07-16 14:14:10 39fb227 Add SQL statement support
• 07-17 16:24:12 be94b0c Set thread context class loader for all plugin factories
• 07-18 18:39:09 71152be Ensure HTTPS enabled when required for internal communication
• 07-18 18:39:09 8b6f0a3 Split out HttpsConfig from HttpServerConfig
• 07-23 12:49:11 feea1ce Add certificate auth doc
• 07-23 12:55:37 ef1c946 Document internal TLS configuration
• 07-23 13:09:10 3c8042b Add views to SQL support
• 07-23 13:10:51 cac8762 Update JDBC connectors to reflect SQL DELETE support

GitHub

• all checked
• 07-23 12:40:12 8522372 Add functions by topic

Grzegorz Kokosiński

• all checked
• 07-02 00:28:53 3013849 Extract complex delete tests to separate test methods
• 07-02 00:28:53 825e6d8 Support metadata DELETE in JDBC connectors
• 07-05 05:08:22 5cacd61 Add missing jol-core SPI dependency
• 07-05 05:08:22 cbea02b Minor cleanup in connectors
• 07-05 05:08:22 da6aa6f Extract utility TypeDeserializerModule
• 07-08 01:25:50 8b2dabd Add grant/revoke schema privileges methods to hive metadata
• 07-13 03:34:38 e84aeb1 Explain statistic bean conflicts in Elastic tests
• 07-14 03:13:03 05bee54 Add additonal commands to cli completion
• 07-14 03:13:03 46e12d9 Fix cli autocompletion
• 07-14 03:13:03 f18a371 Fix cli completion for compound commands
• 07-22 03:33:47 63ca924 Remove JdbcIdentity
• 07-23 05:03:45 13a85ce Extract username from ConnectorIdentity in HdfsAuthentication
• 07-23 05:03:45 e117256 Make UserGroupInformationUtils class public
• 07-23 05:03:45 f41a16c Rename HiveAuthentication to HdfsAuthentication

Karol Sobczak

• all checked
• 07-02 03:22:14 cf45f7b Add missing method call delegation to HiveMaterializedViewMetadata
• 07-05 01:52:10 d5be038 Avoid task sendUpdate until callback clears current request
• 07-06 04:14:31 5ff27ed Fix comment
• 07-06 06:51:06 a9148a9 Introduce SystemTableProvider
• 07-08 00:37:18 12fb514 Support materialized view storage table column casting
• 07-08 00:40:31 cdbf453 Move FastutilSetHelper to trino-plugin-toolkit
• 07-09 04:20:56 05ad467 Move LocalDynamicFiltersCollector to TaskContext
• 07-09 04:20:56 2c2e641 Ensure semijoins dynamic filtering is covered by tests
• 07-09 04:20:56 613bd2f Extract method HttpRemoteTask#triggerUpdate
• 07-09 04:20:56 641aaac Move testJoinDynamicFilteringMultiJoin above helper methods
• 07-09 04:20:56 7854abc Send dynamic filters from coordinator to workers
• 07-12 01:48:51 90e3fd2 Update documentation for enable-coordinator-dynamic-filters-distribution
• 07-14 01:38:41 0177fa2 Make SystemTables final
• 07-14 01:38:41 c235522 Use SystemTableProvider in SystemTableAwareAccessControl
• 07-16 02:35:35 3d9c9de Use PlanBuilder.expression
• 07-16 02:35:35 f35bea0 Relax dynamic filter left expression requirements
• 07-16 02:59:45 6bb9bfc Remove unecessary default implementation in ConnectorPageSourceProvider
• 07-16 02:59:45 b62f03b Remove unused deprecated APIs in ConnectorSplitManager
• 07-16 06:16:06 af1b45f Fix pre-sorted input validation for constants
• 07-19 03:39:27 07c6e73 Implement getCompletedPositions in memory connector
• 07-19 03:39:27 fb10b54 Add getCompletedPositions to ConnectorPageSource
• 07-21 04:12:58 8867aa2 Make DiscreteSet in Domain public
• 07-21 04:14:20 a572e15 Implement getCompletedPositions in ParquetPageSource
• 07-21 05:56:38 d86a2fa Improve positionsArrayToSelectedPositions
• 07-21 06:47:01 df53613 Enable iceberg materialized view tests

Martin Traverso

• all checked
• 07-01 21:17:30 38305b7 Allow exposing custom metrics from the connector
• 07-01 21:17:30 60f66d8 Implement Count and Histogram metrics in trino-plugin-toolkit
• 07-01 21:17:30 d159c6c Move Mergeable interface to SPI
• 07-07 16:10:55 17cf25f Close systemMemoryContext in PartitionedOutputOperator#close()
• 07-07 16:10:55 6bbe9f1 Simplify PagePartitioner inner loop conditional branching
• 07-07 16:10:55 bc1d5ba Implement mayHaveNull() for Dictionary, RLE, and GroupById Blocks
• 07-16 10:14:16 af33a63 Update to aircompressor 0.19
• 07-21 09:57:22 fdb09b1 Fix typo

Piotr Findeisen

• all checked
• 07-16 01:22:35 0648b9a Clarify non-transactional delete is for partitioned tables only
• 07-16 03:23:38 6428a88 Multiply safely
• 07-16 03:23:38 6cfcfc5 Simplify code in BlockAssertions
• 07-16 03:23:38 ade5815 Remove redundant variable
• 07-16 03:23:38 f7fac0e Fix use of deprecated constant
• 07-16 03:23:38 fb9cf4b Remove redundant else
• 07-16 07:18:45 183f78f Fix typo in PinotException#isRetriable method name
• 07-16 07:18:45 64e0c40 Format exception message eagerly
• 07-19 00:24:31 2ceb638 Extract constants
• 07-19 00:24:31 4b4e4f1 Add support for Trino views in Iceberg connector
• 07-19 00:24:31 6c46b26 Filter out Hive system schemas in Iceberg connector
• 07-19 00:24:31 71e0566 Add test for VIEWS and MATERIALIZED VIEWS interoperability
• 07-19 00:24:31 7651769 Add test for Hive and Iceberg views sharing the metastore
• 07-19 00:24:31 adbc8bb Add clarifying comments
• 07-19 04:47:48 0aa32c0 Test Iceberg bucketing stability
• 07-19 04:47:48 ba19a0f Extract Iceberg bucketing method
• 07-20 00:00:32 47163b8 Mark view creator in metastore
• 07-20 03:14:36 b63e636 Fix SHOW CREATE SCHEMA for connectors
• 07-20 03:20:18 03e2323 Add Iceberg bucketing test coverage from spec
• 07-20 03:20:18 5c9ad00 Test Iceberg bucketing of negative decimal values
• 07-20 03:20:18 99171b5 Test Iceberg bucketing for date/time types
• 07-20 03:20:18 bc63717 Test Iceberg bucketing for fixed
• 07-20 06:28:50 5d1523f Remove redundant override
• 07-21 04:14:28 22d6b1b Rename AbstractTestIcebergConnectorTest to current Base*Test convention
• 07-21 04:14:28 3ec60e9 Support range stats on timestamps
• 07-21 04:14:28 4e30d82 Fix reading Iceberg stats when transformation changes type
• 07-21 04:14:28 9884dba Avoid sharing mutable data structure
• 07-21 04:14:28 cf541e6 Propagate stats failures in query runnuers' main() too
• 07-21 04:14:28 d19e581 Remove leftover debug statement
• 07-21 04:57:39 34b19d6 Restore code comment for Iceberg min/max in relation to file format
• 07-21 13:46:41 f15558e Rename constants to match semantics
• 07-21 14:06:44 f412cb4 Add views to SQL support
• 07-22 13:28:22 4ee145c updating hive.rst
• 07-22 23:55:43 8b63cfe Test ALTER TABLE .. RENAME for Iceberg
• 07-22 23:55:43 e2ca8f5 Unblock rename in file metastore
• 07-23 03:02:53 75a4bce Apply attribute to desired folder
• 07-23 03:02:53 bcbf98b Document git attribute meaning
• 07-23 04:46:16 3fa2a18 Fix supportsCreateSchema declaration for Raptor
• 07-23 04:46:16 491dae0 Add clarifying comments
• 07-23 04:46:16 5f20820 Implement BaseConnectorTest for Memory connector
• 07-23 04:46:16 613fcb2 Define testing behavior for add/drop/rename column
• 07-23 04:46:16 6b45e60 Run memory connector tests multi-threaded
• 07-23 04:46:16 724bd6d Throw SkipException in outstanding Kudu tests
• 07-23 04:46:16 8abdbca Test DROP SCHEMA with non-empty schema
• 07-23 04:46:16 d3198b0 Fix supportsCreateSchema, supportsRenameTable declaration for Mongo
• 07-23 06:59:50 87f3aaa Remove redundant parameterization with Iceberg file format in test
• 07-23 06:59:50 acf3c68 Inline redundant variables in tests
• 07-23 06:59:50 ee1fbce Remove disabled test for non-existent feature
• 07-23 06:59:50 efccf52 Increase test coverage
• 07-23 09:52:25 1bf4b65 Fix SHOW CREATE SCHEMA information_schema
• 07-23 09:55:29 40e41de Remove deprecated TupleDomain.transform
• 07-25 12:02:16 28a1cfe Exercise Iceberg applyFilter with non-pushdownable constraint
• 07-25 12:02:16 cd9ca95 Remove incorrect useless code from Iceberg TableStatisticsMaker
• 07-26 03:57:29 30f2ecc Add tests counting metastore accesses for Iceberg

Pratham

• all checked
• 07-12 06:24:04 47301fb Fail Iceberg queries against v2 tables with row level deletes
• 07-12 06:24:04 b602cd8 Fix FileHiveMetastore test version to match TestingTrinoServer
• 07-14 07:05:54 77a24a0 Remove unnecessary locking
• 07-14 08:45:30 a244d3d Wrap stream generated by HadoopInputFile

Yuya Ebihara

• all checked
• 07-07 07:40:08 a35080a Fix README for Trino Verifier
• 07-10 07:04:25 4bab587 Move UUID type to SPI module
• 07-14 17:14:52 4da86ea Fix broken link to BigQuery authentication section
• 07-14 17:22:44 c01b47b Add reusable SQL commands snippet to SQL Server doc
• 07-19 00:11:28 e7a670e Add support for json type in MongoDB
• 07-19 07:26:42 41f8c03 Add DBRef field support for MongoDB connector
• 07-19 07:33:41 e50553d Avoid using deprecated JsonFactory.disable method
• 07-22 18:09:21 d78733c Add JDBC URL parameter for MySQL

Łukasz Osipiuk

• all checked
• 07-06 04:46:26 fcd6b8e Hive cannot read ORC ACID table updated by Trino twice
• 07-07 04:34:17 9d090a5 Inject SessionPropertyManager in DispatchManager
• 07-07 04:34:17 b5217f0 Add SystemSessionPropertiesProvider
• 07-08 14:58:03 1364fbb Make OrcFileWriter number ACID rows correctly across multiple pages
• 07-14 23:54:44 ab9f02d Add Hive Glue statistics tests

[kokosing]

MemSQL, MySQL, Oracle, Phoenix, PostgreSQL, Redshift, SQLServer connectors changes
---
* Partial support for `DELETE` statement where predicate can be fully pushed down to the the remote datasource. {issue}`6287`)

#6287

[sopel39]

General
* Reduce number of requests from coordinator to workers ({issue}`8460`)

#8460

[losipiuk]

Hive
* Ensure generation of unique ROW IDs during UPDATE on ORC ACID transactional tables. 
  Before it was possible that Hive was not able to read the table updated by Trino ({issue}`8268`)

#8268
#8448

[hashhar]

ClickHouse, Druid, MemSQL, MySQL, Oracle, Phoenix, PostgreSQL, Redshift, SQLServer connectors changes
* Add `write.batch-size` connector configuration to control JDBC batch size used during writes.
  It can also be controlled using the `write_batch_size` session property. ({issue}`8434`)

#8434

[sopel39]

SPI
* Cast materialized view storage table columns to match view column types. ({issue}`8408`)

#8408

[losipiuk]

Hive
* Ensure generation of unique ROW IDs during INSERT into ORC ACID transactional tables. 
  Before it was possible that subsequent DELETE from a table would delete more rows than intended ({issue}`8452`)

#8480
#8452

[sopel39]

General:
* Improve query performance by sending collected dynamic filters from coordinator to workers. ({issue}`5183`)

#5183

[kokosing]

CLI
* Fix auto completion when pressing the tab button. ({issue}`8529`)

#8529

[hashhar]

ClickHouse, MemSQL, MySQL, Oracle, Phoenix, PostgreSQL, Redshift, SQLServer connectors changes

* Add new configuration property `insert.non-transactional-insert.enabled` to allow INSERT queries to write directly
  to the target table. This can improve performance in some cases by sacrificing transactional behaviour. It can also
  be controlled using `non_transactional_insert` session property. ({issue}`8496`)

#8496

[hashhar]

MemSQL, MySQL, Phoenix, SQLServer connectors changes

* Fix performance regression of writes in JDBC connectors. ({issue}`8559`)

#8559

[sopel39]

General
* Improve performance of inequality joins where join condition sides have different type. ({issue}`8500`)

#8500

[sopel39]

SPI
* Remove deprecated `ConnectorSplitManager#getSplits` method overrides ({issue}`8569`)

#8569

[sopel39]

General
* Fix query failure when there is a constant predicate on some
  column `col` (e.g `col=1`), followed by  `ORDER BY col` and `LIMIT`. ({issue}`8535`)

#8535

[ebyhr]

MongoDB

* Add support for `json` type. ({issue}`8352`)

#8584

[findepi]

Iceberg
* Add support for Trino views. ({issue}`8540`)

#8540

[sopel39]

SPI
* Introduce `ConnectorPageSource#getCompletedPositions` for tracing physically read positions ({issue}`8524`)

#8524

[ebyhr]

MongoDB

* Support reading MongoDB `DBRef` type. ({issue}`3134`)

#8549

[findepi]

All connectors except of Hive, Iceberg, Memory, Blackhole
* Fix SHOW CREATE SCHEMA. ({issue}`8598`)

#8598

[sopel39]

General:
* Improve performance of queries with `WHERE` predicates. ({issue}`8624`)

#8624

[findepi]

Iceberg
* Fix query failure when joining with a bucketed Iceberg table. ({issue}`7502`)
* Fix query failure when showing stats for a bucketed Iceberg table. ({issue}`8616`)

#7502 #8616

[findepi]

General
* Fix query failure when running `SHOW CREATE SCHEMA information_schema`. ({issue}`8600`)

#8600

[findepi]

SPI
* Remove deprecated `TupleDomain.transform`. ({issue}`8056`)

#8056

[findepi]

Iceberg
* Fix query failure when joining with a partitioned table that has structural columns (`array`, `map` or `row`). ({issue}`8647`)

#8647

[hashhar]

Phoenix 5
* Improve performance of `ORDER BY ... LIMIT` queries on sorted data. ({issue}`8171`)

#8171

[hashhar]

General
* Disallow `DROP SCHEMA` if the schema is not empty. ({issue}`8660`)

#8660

[phd3]

Iceberg Connector
* Fail queries on tables with row-level deletes, since it is not supported. ({issue}`8450`)

#8450

[phd3]

Iceberg Connector
* Fix failures for queries that write tables in Parquet format. ({issue}`5201`)

#5201
#8504

[findepi]

Iceberg
* Improve query planning time. ({issue}`8676`, {issue}`8689`)

#8676, #8689 (for #8675, but that's still not fully done yet)

[martint]

* Improve performance of `IN (<subquery>)` expressions. ({issue}`8639`)

#8639

[kasiafi]

General:
* Improve support for correlated subqueries with GROUP BY or LIMIT and complex correlated filter conditions.

#8554

[sopel39]

General:
* Report cumulative query system memory usage. ({issue}`8615`)

#8615

[martint]

## Elasticsearch

* Add support for assuming an IAM role. ({issue}`8714`)

#8714