Release notes for 365 #9826

martint · 2021-11-02T00:28:39Z

Ashhar Hasan

all checked
11-02 11:48:19 28e8b71 Move comment above relevant code
11-02 11:48:19 fa220ff Replace Optional.isPresent with functional expression
11-03 11:56:03 3fa5f04 Upgrade alluxio-shaded-client to 2.6.2
11-03 16:28:41 7c34f5a Upgrade PostgreSQL driver to 42.3.1
11-03 18:35:55 ef4b887 Add CatalogNameModule to KafkaConnectorFactory
11-05 09:46:15 2db1617 Remove PinotColumn
11-05 09:46:15 4187472 Fix offset handling in passthrough queries
11-05 09:46:15 98cd2bd Rename PinotQuery to PinotQueryInfo
11-05 09:46:15 a78f443 Use JsonCodec when submitting broker request in Pinot
11-05 09:46:15 c1e7173 Upgrade libraries to pinot-0.8.0
11-10 07:10:20 1b5db4a Move ImplementAvgBigint logic from connectors into base jdbc module
11-15 07:03:24 62c7c12 Make PostgreSqlClient.timeWriteFunction() private
11-15 11:59:25 4b4fb69 Clarify token caching
11-17 13:30:32 1562713 Use more functional style to improve code readability
11-22 13:04:21 2fe6b70 Reorder BigQuery view support caveats in docs
11-22 13:36:05 3233388 Fix Pinot test assertion in Java 17
11-22 19:56:51 a6bdf21 Remove redundant test case skipping for MongoDB
11-24 07:55:59 24afd8d Update BigQuery storage protos to v1
11-29 19:02:14 1521016 Support reading DECIMAL columns with unspecified precision in Phoenix connector

Dain Sundstrom

all checked
11-02 22:23:15 37c7d22 Update to latest JWT apis
11-10 21:32:38 2425815 Use IP_ENCODED_AS_HOSTNAME with automatic TLS
11-16 09:59:04 027a82d Add internal view definition classes
11-16 09:59:04 1118307 Remove owner from materialized system table to match the other tables
11-16 09:59:04 230f49f Add ViewInfo to simplify information required when listing views
11-16 09:59:04 32371f7 Notify SystemSecurityMetadata of create, rename, or drop
11-16 09:59:04 44c7be0 Add ownership to SystemSecurityMetadata
11-16 09:59:04 45e4c23 Remove owner from Iceberg materialized system table
11-16 09:59:04 8471cac Add metadata isView to simplify most view usage
11-16 09:59:04 d964e3e Add metadata isMaterializedView to simplify most materialized view usage
11-16 09:59:04 f7aac0d Provide query owner full identity to view and kill query security checks
11-16 23:02:22 29f224f Add QueuedStatementResource timeout for query submission (mem leak fix)
11-17 05:56:20 55f9c7a Add SYSTEM security type to Hive and Iceberg
11-17 05:56:20 c3f6b68 Add ALLOW_ALL and READ_ONLY security to Iceberg
11-19 18:13:26 295da15 Support null owner for connector views when using system security
11-19 18:13:26 4e1b46d Add NO_PRIVILEGES constant to PrincipalPrivileges
11-19 18:13:26 dff92cf Do not set owner or permissions into Hive for system security
11-20 18:53:32 e3cf288 Verify alter owner for Hive system security does not call Hive
11-22 06:38:54 30c6d91 Fix listTablePrivileges when owner is not set
11-24 02:44:51 5b90642 Make event-listener SPI objects fully jackson serializable
11-28 02:53:24 3ac9061 Simplify analyzer setup and usage
11-28 02:53:24 8065aa8 Simplify explain for data definition tasks
11-28 02:53:24 eb5b77d Remove unnecessary DataDefinitionTask execute arguments

David Phillips

all checked
11-09 23:41:23 7ccedcd Enable hive.s3.streaming by default
11-10 21:27:31 016c05e Add schema access requirement to JDBC driver doc
11-16 22:21:45 3e35fc4 Update Maven Wrapper to Maven 3.8.3
11-17 01:33:23 536a63e Use auto-commit flag in Hive connector
11-17 01:33:23 ab22da1 Remove auto-commit flag from ConnectorSession
11-17 01:33:23 bfcb5e8 Migrate Connector.beginTransaction() implementations
11-17 01:33:23 d945f97 Add auto-commit flag to Connector.beginTransaction()
11-17 01:33:23 df8cc87 Inline unnecessary method in HiveMetadataFactory

Grzegorz Kokosiński

all checked
11-02 16:07:09 6f21b71 Remove unnecessary singleton
11-02 20:12:23 dcf8f44 Add owner-based query access rules
11-02 20:12:23 e2bce1f Add group-based query access rules
11-02 20:17:03 835695a Use base SessionPropertiesProvider in Kafka
11-03 16:03:20 6ca69eb Allow Hive and other connectors use system roles
11-03 16:03:20 872ac9b Rename security NONE to ALLOW_ALL
11-03 16:03:20 9e3cb4c Define constants for hive.security options
11-03 16:03:20 d7c932f Extract RaptorSecurity enum
11-09 10:06:49 2dc0e91 Wrap JdbcConnector's ConnectorMetadata
11-10 09:59:43 511c878 Clean up raw types
11-10 10:01:22 200ec0e Update to airlift 209
11-14 21:27:32 02be06c Remove empty delete when not wrapped with exchange
11-14 21:27:32 217d200 Use Table in Insert Statement node
11-14 21:27:32 2ee5ae3 Prefer using TableSchema in StatementAnalyzer#visitDelete
11-14 21:27:32 69c5fcb Pass view refresh WriterTarget as Optional
11-14 21:27:32 790b69c Do not access outerContext when it is absent
11-14 21:27:32 cd05938 Allow to INSERT or DELETE on tables with a row filter
11-16 10:21:37 e4906fd Don't abort the cursor when we're at the end of JDBC resultset
11-17 13:31:37 9d158d4 Remove unused system tables from Hive connector
11-22 11:07:02 0f7cb1f Use method reference
11-22 11:07:02 5025a30 Allow MockConnector to support system roles
11-23 14:09:09 05b483b Test column mask and row filter with hidden columns
11-23 14:09:09 c140d46 Support insert into a table with row filter and hidden columns
11-27 21:29:56 d2fa1fa Improve error message when inserting out of the row filter
11-29 13:33:08 886483e Disable testConcurrentCancellationOnConnectionClose test

Karol Sobczak

all checked
11-02 10:11:12 8c5c5c2 Cover wider range of values in testDecimalBackedByFixedLenByteArray
11-02 11:12:43 1e16b96 Fix formatting of INSERT
11-02 11:12:43 f7d71c1 Fix formatting of DROP TABLE
11-02 13:20:29 9f202ea Benchmark different join build sizes
11-03 11:29:26 50f18a5 Speed up avg by using divideRoundUp vs BigDecimal
11-04 10:15:38 bd41fbd Add order by to tpc-ds query 24
11-04 21:32:42 d1e35c1 Perform one more dynamic partition pruning check
11-05 10:49:22 0fc4494 Store native prefilledValue in ColumnMapping
11-05 10:49:22 4978cd6 Extract columnMappings argument
11-05 10:49:22 aeb753a Run dynamic partition pruning for split locally on a worker node
11-08 10:44:44 488a17f Improve debug info in flaky testDynamicFilters
11-08 10:51:06 320f82e Allow getting metrics after the operator is closed
11-08 11:11:00 9d3cde9 Do not add sources when creating SqlTaskExecution
11-09 16:23:17 8708cd7 Create TaskStatus before stats when reporting taskInfo to coordinator
11-15 23:16:38 b7b515b Fix split accounting in HiveSplitSource when split is pruned
11-17 15:49:41 5697349 Increase split loader concurrency to 64
11-18 11:01:14 017d750 Optimize getPositionsSizeInBytes for fixed width blocks
11-18 11:01:14 e752e96 Avoid branches in DictionaryBlock
11-22 15:03:28 0e4e422 Use GroupByHash#createGroupByHash with Session argument
11-22 16:58:29 7a719f3 Match qualified table name in searchScanFilterAndProjectOperatorStats
11-22 16:58:29 bbbb42a Fix formatting
11-22 16:58:44 51023d8 Restore dynamic partition pruning assertions
11-22 21:23:18 19ad5fa Update dynamic filter ids after un-aliasing
11-23 16:15:18 fab5eef Use smaller load factors for small join hash arrays
11-24 15:18:54 80b8ebb Report read time and nanos for ORC tail reads
11-25 21:36:29 9a824be Implement TDigestHistogram#toString
11-25 21:36:29 b11a241 Add connector metrics to EXPLAIN ANALYZE verbose output
11-25 21:36:29 b47e6b8 Don't allocate a new object when constructing an empty Metrics
11-26 17:30:07 297f1f1 Move FeatureConfig as top package class
11-29 21:19:33 ee80d5e Restore withPlanOptimizersProvider
11-30 11:32:20 5807657 Fix ordering of reads in HdfsParquetDataSource#planRead

Martin Traverso

all checked
11-04 22:37:29 06d7345 Improve SortedPositionLinks performance
11-04 22:37:29 4b253dd Make stateless helper methods static in SortedPositionLinks
11-04 22:37:29 6c20a1a Add test to verify BenchmarkInequalityJoin works
11-04 22:37:29 816596b Remove size() method from PositionLinks
11-04 22:37:29 a716254 Make LocalQueryRunner initial transaction autocommit
11-05 00:08:39 02f5c7e Set standard error duration for task cleanup requests
11-05 00:08:39 12b3efc Accept Executor interface in StageStateMachine
11-05 00:08:39 187e7f6 Refactor HttpRemoteTask
11-05 00:08:39 374f9e1 Remove mergable check from PlanNodeStats
11-05 00:08:39 3bbb5d3 Ignore task abort request failure if the task is finished
11-05 00:08:39 46984e1 Decrease visibility of scheduleRequestIfNecessary
11-05 00:08:39 8a5ad97 Fail with transport exception on results abort failure
11-05 00:08:39 92387da Check response status of the abort task results request
11-05 00:08:39 a7e7664 Reduce testTargetMaxFileSize memory footprint
11-05 00:08:39 b434199 Decouple collocated join from grouped execution
11-05 00:08:39 c3f73b8 Preserve internal task status for tasks marked as failed
11-05 00:08:39 da2cd0a Ensure cached query results are discarded
11-05 00:08:39 ea11bed Don't try to abort unreachable tasks
11-08 20:04:14 4ca945f Implement contains function for IP address
11-11 18:42:13 1b14269 Fix handling of null pages in getDomain for column index
11-11 18:42:13 275f9e3 Make TupleDomainParquetPredicate#getDomain static
11-12 22:15:41 1535550 Reduce overhead of PipelineStatus construction
11-12 22:15:41 387c444 Avoid DataSize.succinctBytes in operator summaries
11-12 22:15:41 43704ae Fix incorrect handling of System.nanoTime() in DriverContext
11-12 22:15:41 95bb858 Avoid unnecessary intermediate collections in PipelineContext
11-12 22:15:41 a748831 Batch merge running operator stats in PipelineContext
11-12 22:15:41 c9c711e Opportunistically reuse instance in OperatorStats#summarize()
11-12 22:15:41 cfd0c24 Avoid intermediate collection creation in TaskStats creation
11-12 22:15:41 d676634 Remove redundant traversal of OperatorStats
11-12 22:15:41 e703ddd Avoid redundant copies in PipelineStats#summarize
11-12 22:15:41 f54f076 Avoid redundant copies in TaskStats#summarize
11-16 12:21:56 1c23c69 Add syntax for querying tables as if it were a specific time or version
11-17 14:03:46 5f79547 Detect no nulls selected in RowBlock#copyPositions
11-17 14:03:46 723ea82 Use ColumnarRow#mayHaveNull() in ReaderProjectionsAdapter
11-17 14:03:46 764ba1e Add ColumnarRow#mayHaveNull()
11-17 14:03:46 c4e8a82 Use ColumnarRow#mayHaveNull() in HiveUpdatablePageSource#deleteRows
11-17 14:03:46 d1a5d2c Improve ColumnarRow dictionary block handling
11-17 14:03:46 d1d741e Use ColumnarRow#mayHaveNull() in hive StructCoercer
11-18 22:08:48 08b1097 Simplify negative byte check in getShortDecimalValue
11-18 22:08:48 8d23397 Fix reading fixed_len_byte_array of length 9-16 in parquet
11-19 11:55:44 e656ec4 Improve performance when constructing QueryId instance
11-20 11:05:54 4605d48 Fix iceberg build
11-23 01:29:30 d90c1b7 fix code style warning
11-23 01:29:30 f025950 Fix Elastic Cloud Instance

Piotr Findeisen

all checked
11-02 12:23:48 16dfce0 Bump Iceberg version to 0.12.0
11-02 15:39:40 deaff22 Rename hive.dynamic-filtering-probe-blocking-timeout config
11-03 13:57:54 0bbb73c Predicate pushdown for Paruqet int64 timestamps
11-03 13:57:54 ff2b13f Pass ColumnDescriptor through TupleDomainParquetPredicate
11-03 14:03:51 2e9c40e Support ORC Iceberg boolean statistics
11-04 10:18:07 aaa54e0 Allow disabling of Iceberg table statistics
11-04 15:20:01 439d386 Repartition data when writing to Iceberg table
11-04 15:20:01 57a144f Consolidate Iceberg temporal transformations
11-04 15:20:01 650ebc5 Remove unused Iceberg partitioning provider
11-04 15:20:01 863da2b Add value transform to Iceberg ColumnTransform
11-04 15:20:01 9e7e15b Construct reused hash object for Iceberg bucketing
11-04 15:20:01 b4c10be Inline or rename misused variable assignment
11-04 15:20:01 c631ce2 Simplify test queries counting files
11-04 15:20:01 efa088e Consolidate ColumnTransform definition
11-04 15:20:01 f56342c Fix typo
11-04 15:20:01 f709aae Simplify Iceberg bucket transform definition
11-04 15:23:01 d10cb87 Rename rule consuming preferred write partitioning
11-08 13:14:07 1d771ee Speed up translation of IN / NOT IN to tuple domain
11-08 13:14:07 38ac9dd Remove static import of TupleDomain.withColumnDomains
11-08 13:14:07 815c0d3 Improve tests for IN, NOT IN domain translation
11-08 13:14:07 8e67fa4 Import Arrays.asList statically
11-08 13:14:07 a5c6dad Add assertPredicateTranslates with remaining expression
11-09 10:02:14 65b7d17 Extract getSchemaPath in FileHiveMetastore
11-09 10:02:14 68cd03a Use an enum for schema types in FileHiveMetastore
11-09 10:02:14 9192047 Inline null check in FileHiveMetastore constructor
11-09 10:02:14 ae48b3f Extract file names as constants in FileHiveMetastore
11-09 10:02:14 f36b894 Move FileHiveMetastore path getters near end of file
11-09 10:02:14 ffac206 Replace deprecated function in TestCreateDropSchema
11-09 13:07:19 ba77719 Fix raw types use in Metrics
11-11 22:13:04 676aee4 Remove invalid test assumption and comment
11-11 22:13:04 8c6b950 Move @dataProvider under test method
11-11 22:13:04 9b40ead Add stats-based writer partitioning for identity-partitioned Iceberg table
11-11 22:13:04 dd286ed Extract source relation and add Session parameter to test helper
11-13 11:24:49 a836e7c Declare required tables in AbstractTestJoinQueries
11-22 11:35:38 304b99f Test partition scan limit for multi-column partitioning
11-22 11:35:38 4a39c41 Inline test method
11-22 12:28:37 16be328 Inject type analyzer into SqlQueryExecution instead of creating it
11-22 12:28:37 a0fb364 Extract LayoutConstraintEvaluator from PushPredicateIntoTableScan
11-22 12:28:37 c973251 Filter out splits that do not match predicates not expressible by tuple domain
11-22 13:17:17 0682db3 Make TypedSet.add return a boolean
11-22 13:17:17 0f78fa9 Use boolean return from TypedSet.add instead of TypedSet.contains
11-22 13:17:17 131f91b Add explicit int/long cast in TypedSet
11-22 13:17:17 2fe933f Rename blockBuilder parameter in TypedSet to elementBlock
11-22 13:17:17 3f4daba Add javadoc for TypedSet
11-22 13:17:17 4fa5116 Return early from TypedSet.add when attempting to add a second null
11-22 13:17:17 6d401c3 Make TypedSet constructor private
11-22 13:17:17 dbe03a7 Rename factory function for creating unbounded TypedSets
11-22 13:27:24 763c77e Support reading parquet int64 timestamp as bigint
11-22 13:27:24 92f2452 Renamed utcMillis to epochMillis in Int64TimestampMillisColumnReader
11-22 13:27:24 f27d4eb Refactored TestTimestamp by adding new test-function for reading with different types
11-22 14:56:45 254bacd Remove redundant test case skipping for SQL Server
11-23 10:08:00 0f3b4ed Use fast path in array_except when the right-hand array is empty
11-23 10:08:00 5ea20df Fix expected size of result array in array_except
11-23 10:08:00 6ab78e6 Use more descriptive variable names in MapFromEntriesFunction
11-23 10:08:00 8071657 Use more descriptive variable names in MultimapFromEntriesFunction
11-23 10:08:00 c68bda7 Rename variables in MapToMapCast
11-23 10:08:00 d224e0f Don't do extra work in array_union
11-23 10:08:00 ef64e05 Use just one BlockBuilder to construct result in array_distinct
11-23 10:41:21 0281721 Convert discrete domain to Iceberg IN expression
11-23 10:41:21 646591b Remove redundant cast to SortedRangeSet
11-23 10:45:11 88ef2ca Add Iceberg ORC timestamp stats implementation using ms precision
11-23 16:22:35 e0a0496 Remove redundant flexible assertions
11-24 12:55:03 35c4588 Document transactions support in Hive connector
11-24 17:25:23 9c1e32b Update TPCH plugin to normalize schema name
11-24 17:26:35 ec62b27 Remove probably obsolete @flaky annotation
11-24 17:39:31 ea633f9 Consistently reject empty schema location
11-24 20:17:07 302d942 Run testIncorrectIcebergFileSizes for ORC too
11-24 20:17:07 83d7d12 Fix file_size_in_bytes in Iceberg manifest for ORC
11-25 12:15:49 d66b0f9 Supplement defensive copying in HiveTableHandle constructor
11-25 12:15:49 d99d2a6 Remove redundant Optional use from HiveTableHandle.constraintColumns
11-25 12:15:49 e553381 Remove redundant Optional use from HiveTableHandle.projectedColumns
11-25 14:31:47 8274da8 Allow ignoring schema location cleanup
11-25 20:45:42 ef5084c Avoid dynamic filter current predicate computation when not used
11-26 17:26:58 28d4fcd Revert "Only delete non-empty directories when dropping Hive schemas"
11-26 17:26:58 7c031d4 Revert "Allow ignoring schema location cleanup"
11-26 22:15:34 884e450 Cleanup javadoc tags
11-26 22:37:30 5c98059 Upgrade Error Prone to 2.10.0
11-26 22:54:28 530c2c9 Report redundant exclusions in assertAllMethodsOverridden
11-26 22:54:28 84ee460 Remove redundant method exclusion from test
11-26 22:54:28 c6d6a21 Implement new methods in AllowAllSystemAccessControl
11-29 09:16:50 d5b100a Access test resources with proper validation

Pratham

all checked
11-08 21:52:36 2c37757 Add info on query-partition-filter to Hive
11-15 04:05:20 1a7d82e Fix rollback_to_snapshot failure caused by stale metadata
11-20 01:27:12 6055c60 Add dereference pushdown section
11-20 01:30:50 21f9c83 Fix handling of ORC projection layouts where field has been renamed
11-20 01:30:50 5fc2d0d Implement dereference pushdown for the Iceberg connector
11-20 01:30:50 617808f Update ReaderProjectionsAdapter parameters
11-20 01:30:50 6cdce6c Refactor Iceberg ColumnIdentity#children to be mapped by field id
11-20 01:30:50 a93bbe3 Test Iceberg tables with ambiguous schemas
11-20 01:30:50 e3a5842 Generalize OrcReader's ProjectLayout
11-22 23:28:18 4563866 Add clarifying comment
11-29 16:35:42 437230b Update table name in test

Praveen Krishna

all checked
11-10 07:06:37 1250def Remove redundant check on bloom filter fpp
11-10 07:06:37 1f781db Remove exclusion for HashtableContains in ErrorProne compiler
11-10 07:06:37 9c06e33 Fix OrcWriterOptions mapping from Hive TableProperties
11-19 07:29:09 89e7c9b Avoid creating splits for empty files in Hive
11-19 07:30:04 44c3854 Use Classloader safe implementation of RecordSetProvider for Phoenix
11-19 07:30:04 85fd6b6 Bump docker images version to 52
11-23 13:43:02 bead755 Add docs for Kerberos delegation in JDBC client
11-26 06:58:10 040fa59 Extract configuration to HydraIdentityProvider
11-26 06:58:10 430b095 Add Product Tests for OAuth2 with secure proxy
11-26 06:58:10 a1b5a30 Add Product Tests for OAuth2 with HTTP Proxy
11-26 06:58:10 ff0d93b Remove redundant dependencies from EnvSinglenodeOauth2

Yuya Ebihara

all checked
11-04 13:41:29 83571b6 Set SMP as 1 in scylladb container
11-05 01:01:52 f0bf9e9 Fix a small typo
11-05 01:16:09 338425d Implement TRUNCATE TABLE in base JDBC connector
11-05 01:16:09 9acc19b Support TRUNCATE TABLE statement
11-05 06:11:31 c1ab1dd Support user defined type in Cassandra
11-09 11:15:15 5cda4cd Do not find tables in information_schema in BigQuery
11-10 04:59:00 7e8cc84 Remove duplicate licenses
11-10 13:53:44 6d2458a Support reading BIGNUMERIC type in BigQuery
11-11 08:30:08 9f915ca Support reading empty numeric value as NULL in Elasticsearch
11-15 10:40:25 884bc09 Use ifPresentOrElse in BigQueryMetadata
11-15 10:40:25 a0d3dc2 Ignore disappeared datasets during listing tables in BigQuery
11-16 10:22:16 4db9a48 Populate TPCH tables in SqlServerQueryRunner.main method
11-16 10:22:16 51c4293 Use MSSQLServerContainer.acceptLicense in tests
11-16 10:22:16 e448eba Add smoke test to SQL Server connector with 2017 and 2019 image
11-19 10:04:10 14a7e17 Disable predicates pushdown to ClickHouse FLOAT type
11-19 10:04:10 4df037a Disable predicates pushdown to MemSQL REAL type
11-19 10:04:10 52deac3 Remove unused method in MySQL mapping test
11-19 10:04:10 75878e0 Disable predicates pushdown to MySQL REAL type
11-26 08:37:04 2482610 Extract BigQuery view materialization cache to separate class
11-26 08:37:04 2f25645 Refactor BigQuery to allow non-static credential suppliers

kasiafi

all checked
11-16 10:14:18 c63e06f Support accumulator state copying

Łukasz Osipiuk

all checked
11-03 12:31:49 0aaa6f4 Use Rule to prune unreferenced source columns for TableExecuteNode
11-04 11:36:09 23b1c54 Fix flaky TestIcebergOrcConnectorTest.testAllAvailableTypes
11-04 12:45:46 4084368 Add javadoc for DeterminePreferredWritePartitioning
11-05 10:07:50 1978345 Drop unused port mappings
11-05 10:07:50 5f8272e Drop unneded String.format
11-05 10:07:50 e9cd44b Add endpoint debug logs on HiveHadoop startup
11-05 15:51:45 43e5836 Tests Trino with Spark-created Iceberg V2 tables
11-05 15:51:45 7432b01 Bump docker images version to 51
11-08 09:07:38 b40f5f6 Add docs on running checkstyle locally
11-08 11:02:49 6eec624 Correct what tables optimize can be used with
11-08 21:05:13 3935f5f Make partition overwrite assertion use exact rows
11-08 21:05:13 f7c87f8 Fix partition overwrite assertion
11-10 13:52:12 19076b6 Add test for default session properties in transaction
11-10 13:52:12 66948b3 Inject AccessControl to SessionPropertyDefaults
11-10 13:52:12 7053d72 Remove redundant argument in SessionPropertyManager.validateCatalogSessionProperty
11-10 13:52:12 a671947 Allow session properties override once in transaction
11-10 13:52:12 e173be4 Add allow all access control manager for tests
11-10 15:01:54 56f77ee Add missing method from interface
11-18 10:24:39 244b4d0 Upgrade trino hadoop-apache to 3.2.0-16
11-18 21:49:38 02ac19e Handle null statistics in Iceberg Dynamic Filtering
11-19 13:04:28 92d7eea Account memory for OrcDeletedRows
11-19 13:05:01 f5bff8e Add redirections scaffolding to HiveMetadata
11-20 21:52:22 a89d7c1 Only delete non-empty directories when dropping Hive schemas
11-22 11:39:22 61c220b Construct RLE block with proper types for Parquet with missing struct fields
11-23 09:03:45 3dc0019 Correct S3 credentials provider property
11-23 11:11:26 cd20327 Remove obsolete comment
11-26 11:48:23 c94f3b9 Update memory usage for DeletedRows as it is loaded
11-26 11:50:17 ef69360 Clarify javadoc

The text was updated successfully, but these errors were encountered:

sopel39 · 2021-11-02T10:14:12Z

General:
* Fix ``PREPERATE`` statements for ``DROP/INSERT`` when table or schema name contains special characters. ({issue}`9823`)

#9823

kokosing · 2021-11-02T19:13:43Z

Security
* Add group-based and owner-based query access rules to file based system access control. ({issue}`9811`)

#9811

sopel39 · 2021-11-03T10:30:16Z

Hive
* Improve performance of decimal ``avg`` aggregation. ({issue}`9738`)

#9738

findepi · 2021-11-03T12:59:56Z

Hive, Iceberg
* Improve query performance over Parquet data with timestamp encoded using `int64` representation in Parquet. ({issue}`9414`)

#9414

findepi · 2021-11-04T14:22:17Z

Iceberg
* Reduce resource consumption and create bigger files when writing to an Iceberg table with partitioning. Bigger files are more efficient to query later. ({issue}`9826`)

#9826

sopel39 · 2021-11-04T20:33:58Z

Hive
* Improve dynamic partition pruning efficiency. ({issue}`9866`, `9869`)

#9866, #9869

ebyhr · 2021-11-05T00:26:42Z

## General
* Add support for [`TRUNCATE TABLE`](/sql/truncate). ({issue}`8921`)

## ClickHouse
* Add support for truncating tables. ({issue}`8921`)

## Druid
* Add support for truncating tables. ({issue}`8921`)

## MemSQL
* Add support for truncating tables. ({issue}`8921`)

## MySQL
* Add support for truncating tables. ({issue}`8921`)

## Oracle
* Add support for truncating tables. ({issue}`8921`)

## PostgreSQL
* Add support for truncating tables. ({issue}`8921`)

## Redshift
* Add support for truncating tables. ({issue}`8921`)

## SQL Server
* Add support for truncating tables. ({issue}`8921`)

#8932

ebyhr · 2021-11-05T05:13:23Z

## Cassandra
* Support reading user defined types in Cassandra. ({issue}`147`)

#9685

hashhar · 2021-11-05T08:49:25Z

## Pinot
* Update Pinot connector to be compatible with versions >= 0.8.0 and drop support for older versions. ({issue}`9098`)

sopel39 · 2021-11-08T09:52:24Z

SPI
* Collect connector metrics after page source ``ConnectorPageSource`` is closed. ({issue}`9615`)

#9615

sopel39 · 2021-11-08T10:14:01Z

General
* Make query final statistics more accurate. ({issue}`9888`, `9913`)

#9888, #9913

findepi · 2021-11-08T12:14:41Z

General
* Improve query planning performance for queries containing large `IN` predicates. ({issue}`9874`)

#9874

martint · 2021-11-08T19:40:34Z

* Add {func}`contains` to check whether a CIDR contains an IP address. ({issue}`9654`)

#9654

ebyhr · 2021-11-10T12:58:23Z

## BigQuery
* Support reading `bignumeric` type whose precision is less than or equal to 38. ({issue}`9882`)

#9882

dain · 2021-11-10T20:54:04Z

## Security
* Fix internal communication automatic TLS on Java 17. ({issue}`9821`)
* Use internal names for discover client when automatic TLS is enabled for internal communications.  This allows `discovery.uri` to configured using a normal DNS name like `https://coordinator.trino`, and still use automatic TLS certificates. ({issue}`9821`)

#9821

ebyhr · 2021-11-11T07:31:36Z

## Elasticsearch
* Fix failure when empty values exist in numeric fields. ({issue}`9939`)

#9939

kokosing · 2021-11-14T20:28:47Z

General
 - Allow to execute ``INSERT`` or ``DELETE`` statements on tables restricted with a row filter. ({issue}`8856`)

#8856

erichwang · 2021-11-16T22:04:31Z

General
* Fix minor memory leak when queries are abandoned during the initial query submission phase

#9962

sopel39 · 2021-11-17T14:50:22Z

Hive:
* Improve query performance on partitioned tables or tables with small files. ({issue}`9979`)

#9979

ebyhr · 2021-11-19T09:08:46Z

## MySQL
* Fix incorrect query results when query contains predicates on `real` type columns. ({issue}`9998`)

## MemSQL
* Fix incorrect query results when query contains predicates on `real` type columns. ({issue}`9998`)

## ClickHouse
* Fix incorrect query results when query contains predicates on `real` type columns. ({issue}`9998`)

#9998 #10000

findepi · 2021-11-22T11:30:26Z

Iceberg
* Improve query performance for certain queries with complex predicates. ({issue}`9309`)

SPI
* Allow split manager to filter splits based on a predicate not expressible as a `TupleDomain`. ({issue}`7608`)

#7608 #9309 #9830

ebyhr · 2021-11-23T01:38:25Z

## BigQuery
* Fix failure when a schema is dropped during listing tables. ({issue}`9954`)

#9954

sopel39 · 2021-11-23T15:16:03Z

General:
* Improve join performance for joins with small build side. ({issue}`9851`)

#9851

sopel39 · 2021-11-24T14:20:01Z

Hive
* Fix reporting of ORC files read bytes. ({issue}`10048`)

#10048

findepi · 2021-11-24T19:18:18Z

Iceberg
* Write correct `file_size_in_bytes` in manifest when creating new ORC files. ({issue}`9810`)

#9810 #10042

sopel39 · 2021-11-25T20:37:27Z

General:
* Report connector metrics in `EXPLAIN ANALYZE VERBOSE`. ({issue}`9858`)

#9858

hashhar · 2021-11-29T18:09:28Z

## Phoenix
* Support reading ``DECIMAL`` columns from Phoenix with unspecified precision or scale. ({issue}`9795`)

Praveen2112 · 2021-11-30T13:26:53Z

Phoenix: 
* Fix query failures when reading Phoenix tables. ({issue}`9151`)

#9942

Praveen2112 · 2021-11-30T13:32:36Z

Hive:
* Fix bug that skips writing bloom filters for ORC files. ({issue}`9792`)

#9774

Praveen2112 · 2021-11-30T13:35:26Z

Hive:
* Allow reading empty files.

#9929

Praveen2112 · 2021-11-30T13:37:31Z

Security:
* Allow configuring HTTP proxy for OAuth2 authentication.

#9920
#10069

Praveen2112 · 2021-11-30T14:22:42Z

Kudu:
* Avoid scanner time-out issues when reading Kudu tables. ({issue}`7250`)

#9947

findepi · 2021-11-30T14:31:38Z

Hive
* Fix incorrect rows being deleted when deleting from transactional table that has original files. ({issue}`10095`)

#10095

phd3 · 2021-11-30T15:17:05Z

* Iceberg
- Fix failure caused by stale metadata in the `rollback_to_snapshot` procedure. ({issue}`9221`)
- Improve performance for queries on nested data through dereference pushdown. ({issue}`8129`)

findepi · 2021-12-01T08:23:37Z

Hive
* Fix delete and update failure when changing a table after a major compaction.

#10120

kasiafi · 2021-12-01T09:04:11Z

General:
* Fix incorrect results when casting bigint values to bounded varchar type. ({issue}`552`)

#10090

sopel39 · 2021-12-01T10:29:40Z

Hive:
* Fix ``hive.dynamic-filtering.wait-timeout`` not having any effect. ({issue}`10106`)

#10106

findepi · 2021-12-01T13:40:41Z

Iceberg
* Use ZSTD by default.

#10058 #10045

findepi · 2021-12-02T09:10:32Z

Iceberg
* Add support for storing and reading UUID nested in row, array or map type. ({issue}`9918`)

#9930 #9918

sopel39 · 2021-12-02T09:43:46Z

General
* Report operator input distribution in ``EXPLAIN ANALYZE VERBOSE``. ({issue}`10133`)

#10133

ebyhr · 2021-12-02T12:43:44Z

## MongoDB
* Support connecting to MongoDB clusters via `mongodb.connection-url` config property. `mongodb.seeds` and `mongodb.credentials` properties are now deprecated. ({issue}`9819`)

#9819

sopel39 · 2021-12-02T15:45:03Z

General
* Improve CPU utilization by adjusting ``task.concurrency`` automatically for number of physical cores. ({issue}`10088`)

#10088

kokosing · 2021-12-02T20:57:01Z

Security
* Use Kerberos operating system ticket cache if keytab file is not provided in JDBC and CLI for Kerberos authentication. ({issue}`8987`)
* Do not allow to specify incorrect path for `KerberosKeytabPath` JDBC property. ({issue}`8982`)
* Do not allow to specify incorrect path for `--krb5-keytab-path` Trino CLI argument. ({issue}`8982`)

phd3 · 2021-12-03T00:20:44Z

* Iceberg Connector:
- Use Iceberg's `schema.name-mapping.default` table property for scanning files 
  with missing Iceberg ids. This aligns Trino behavior on migrated files with the 
  Iceberg spec. ({issue}`9959`)

dain · 2021-12-03T01:21:28Z

General
* Remove `owner` column from the `metadata.materialized_views` table.

Iceberg
* Add  read-only security option which can be enabled setting the configuration`iceberg.security=read-only`.

ebyhr · 2021-12-03T06:34:05Z

## CLI
* Use HTTPS in CLI when using port 443. ({issue}`8798`)

#8806

Praveen2112 · 2021-12-03T10:47:36Z

Hive
* Allow `REVOKE GRANT OPTION` to revoke only the granting instead of revoking entire privilege.

#10094

findepi · 2021-12-03T10:48:11Z

iceberg
* Change schema of `$partitions` system table to avoid conflicts when table name contains a column named `row_count`, `file_count` or `total_size`, or when a column is used for partitioning for part of table data, and it not used for partitioning in some other part of the table data. ({issue}`9519`, {issue}`8729).

#8729 #9519 #10142

phd3 · 2021-12-03T16:02:18Z

* General
- Remove legacy column-pruning optimizer by removing the config property 
  `optimizer.iterative-rule-based-column-pruning`. ({issue}`9564`)

I'm not sure if we need this though. cc @kasiafi @martint

kasiafi · 2021-12-03T17:19:58Z

I would say it the other way: remove the config property ... so that the legacy optimizer is no more available.

kasiafi · 2021-12-03T20:08:54Z

General
* Support aggregate functions in row pattern recognition context.

#8738

hashhar · 2021-12-03T21:22:41Z

## PostgreSQL
* Add experimental support for range predicate pushdown on string columns. It can be enabled
  by setting the `postgresql.experimental.enable-string-pushdown-with-collate` catalog configuration
  property or the corresponding `enable_string_pushdown_with_collate` session property to `true`. ({issue}`9746`)

electrum · 2021-12-03T21:44:35Z

# Hive connector

* Enable `hive.s3.streaming` by default. ({issue}`9715`)

#9715

martint added the release-notes label Nov 2, 2021

martint added this to the 365 milestone Nov 2, 2021

findepi pinned this issue Nov 3, 2021

martint mentioned this issue Dec 2, 2021

Add 365 release notes #10145

Merged

martint closed this as completed in #10145 Dec 4, 2021

martint unpinned this issue Dec 4, 2021

trinodb deleted a comment Jul 16, 2023

Release notes for 365 #9826

Release notes for 365 #9826

Comments

martint commented Nov 2, 2021 • edited by electrum Loading

Ashhar Hasan

Dain Sundstrom

David Phillips

Grzegorz Kokosiński

Karol Sobczak

Martin Traverso

Piotr Findeisen

Pratham

Praveen Krishna

Yuya Ebihara

kasiafi

Łukasz Osipiuk

sopel39 commented Nov 2, 2021

kokosing commented Nov 2, 2021

sopel39 commented Nov 3, 2021 • edited Loading

findepi commented Nov 3, 2021

findepi commented Nov 4, 2021

sopel39 commented Nov 4, 2021 • edited Loading

ebyhr commented Nov 5, 2021

ebyhr commented Nov 5, 2021

hashhar commented Nov 5, 2021

sopel39 commented Nov 8, 2021

sopel39 commented Nov 8, 2021 • edited Loading

findepi commented Nov 8, 2021

martint commented Nov 8, 2021

ebyhr commented Nov 10, 2021

dain commented Nov 10, 2021

ebyhr commented Nov 11, 2021

kokosing commented Nov 14, 2021

erichwang commented Nov 16, 2021

sopel39 commented Nov 17, 2021

ebyhr commented Nov 19, 2021

findepi commented Nov 22, 2021

ebyhr commented Nov 23, 2021

sopel39 commented Nov 23, 2021

sopel39 commented Nov 24, 2021

findepi commented Nov 24, 2021

sopel39 commented Nov 25, 2021

hashhar commented Nov 29, 2021

Praveen2112 commented Nov 30, 2021

Praveen2112 commented Nov 30, 2021

Praveen2112 commented Nov 30, 2021

Praveen2112 commented Nov 30, 2021

Praveen2112 commented Nov 30, 2021

findepi commented Nov 30, 2021

phd3 commented Nov 30, 2021 • edited Loading

findepi commented Dec 1, 2021

kasiafi commented Dec 1, 2021

sopel39 commented Dec 1, 2021

findepi commented Dec 1, 2021 • edited Loading

findepi commented Dec 2, 2021

sopel39 commented Dec 2, 2021

ebyhr commented Dec 2, 2021

sopel39 commented Dec 2, 2021 • edited Loading

kokosing commented Dec 2, 2021

phd3 commented Dec 3, 2021 • edited Loading

dain commented Dec 3, 2021

ebyhr commented Dec 3, 2021

Praveen2112 commented Dec 3, 2021

findepi commented Dec 3, 2021

phd3 commented Dec 3, 2021

kasiafi commented Dec 3, 2021

kasiafi commented Dec 3, 2021

hashhar commented Dec 3, 2021

electrum commented Dec 3, 2021

martint commented Nov 2, 2021 •

edited by electrum

Loading

sopel39 commented Nov 3, 2021 •

edited

Loading

sopel39 commented Nov 4, 2021 •

edited

Loading

sopel39 commented Nov 8, 2021 •

edited

Loading

phd3 commented Nov 30, 2021 •

edited

Loading

findepi commented Dec 1, 2021 •

edited

Loading

sopel39 commented Dec 2, 2021 •

edited

Loading

phd3 commented Dec 3, 2021 •

edited

Loading