diff --git a/changelog/21.0/21.0.0/changelog.md b/changelog/21.0/21.0.0/changelog.md index 5b917f95867..32b09e1344e 100644 --- a/changelog/21.0/21.0.0/changelog.md +++ b/changelog/21.0/21.0.0/changelog.md @@ -14,7 +14,8 @@ * Fix golang upgrade workflow [#16890](https://github.com/vitessio/vitess/pull/16890) #### Cluster management * Use default schema reload config values when config file is empty [#16393](https://github.com/vitessio/vitess/pull/16393) - * fix: Use authentication_policy to specify default auth plugin for MySQL 8.4 [#16426](https://github.com/vitessio/vitess/pull/16426) + * fix: Use authentication_policy to specify default auth plugin for MySQL 8.4 [#16426](https://github.com/vitessio/vitess/pull/16426) + * [Direct PR] [release-21.0] Revert Errant GTID detection in vttablets [#17019](https://github.com/vitessio/vitess/pull/17019) #### Docker * Fix the install dependencies script in Docker [#16340](https://github.com/vitessio/vitess/pull/16340) #### Documentation @@ -90,7 +91,8 @@ * Migrate Workflow: Scope vindex names correctly when target and source keyspace have different names [#16769](https://github.com/vitessio/vitess/pull/16769) * VReplication: Support both vindex col def formats when initing target sequences [#16862](https://github.com/vitessio/vitess/pull/16862) * [release-21.0] fix: Infinite logs in case of non-existent stream logs (#17004) [#17014](https://github.com/vitessio/vitess/pull/17014) - * [release-21.0] VReplication: Support reversing read-only traffic in vtctldclient (#16920) [#17015](https://github.com/vitessio/vitess/pull/17015) + * [release-21.0] VReplication: Support reversing read-only traffic in vtctldclient (#16920) [#17015](https://github.com/vitessio/vitess/pull/17015) + * [release-21.0] Use proper zstd decoder pool for binlog event compression handling (#17042) [#17045](https://github.com/vitessio/vitess/pull/17045) #### VTAdmin * VTAdmin: Upgrade websockets js package [#16504](https://github.com/vitessio/vitess/pull/16504) * VTAdmin: Fix serve-handler's path-to-regexp dep and add default schema refresh [#16778](https://github.com/vitessio/vitess/pull/16778) @@ -128,7 +130,8 @@ * [release-21.0] Flakes: Address flakes in TestMoveTables* unit tests (#16942) [#16951](https://github.com/vitessio/vitess/pull/16951) #### Docker * Docker: Update node vtadmin version [#16147](https://github.com/vitessio/vitess/pull/16147) - * Fix `docker_lite_push` make target [#16662](https://github.com/vitessio/vitess/pull/16662) + * Fix `docker_lite_push` make target [#16662](https://github.com/vitessio/vitess/pull/16662) + * [release-21.0] Build `vttestserver` in GHA and send Slack message on Docker Build failure (#16963) [#17047](https://github.com/vitessio/vitess/pull/17047) #### General * [main] Upgrade the Golang version to `go1.22.4` [#16062](https://github.com/vitessio/vitess/pull/16062) * [main] Upgrade the Golang version to `go1.22.5` [#16319](https://github.com/vitessio/vitess/pull/16319) @@ -169,6 +172,8 @@ * [release-21.0] Add missing changelog for PR #16852 (#17002) [#17006](https://github.com/vitessio/vitess/pull/17006) #### General * release notes: update dml related release notes [#16241](https://github.com/vitessio/vitess/pull/16241) +#### Query Serving + * [Direct PR][release note]: Atomic Distributed Transaction [#17079](https://github.com/vitessio/vitess/pull/17079) #### VReplication * Release docs: Add vreplication related entries to the v20 summary [#16259](https://github.com/vitessio/vitess/pull/16259) #### VTTablet @@ -183,7 +188,8 @@ * Prefer replicas that have innodb buffer pool populated in PRS [#16374](https://github.com/vitessio/vitess/pull/16374) * Allow cross cell promotion in PRS [#16461](https://github.com/vitessio/vitess/pull/16461) * Fix: Errant GTID detection on the replicas when they set replication source [#16833](https://github.com/vitessio/vitess/pull/16833) - * [release-21.0] [Direct PR] Add RPC to read the reparent journal position [#16982](https://github.com/vitessio/vitess/pull/16982) + * [release-21.0] [Direct PR] Add RPC to read the reparent journal position [#16982](https://github.com/vitessio/vitess/pull/16982) + * [Direct PR] [release-21.0] Augment `PrimaryStatus` to also send Server UUID [#17032](https://github.com/vitessio/vitess/pull/17032) #### Docker * Remove the `bootstrap` dependency on the Docker images we ship [#16339](https://github.com/vitessio/vitess/pull/16339) #### Evalengine @@ -374,7 +380,7 @@ * Support passing filters to `discovery.NewHealthCheck(...)` [#16170](https://github.com/vitessio/vitess/pull/16170) #### VTTablet * `txthrottler`: move `ThrottlerInterface` to `go/vt/throttler`, use `slices` pkg, add stats [#16248](https://github.com/vitessio/vitess/pull/16248) - * Deprecate vttablet metrics `QueryCacheXX` and rename to `TabletQueryPlanCacheXX` [#16289](https://github.com/vitessio/vitess/pull/16289) + * Deprecate vttablet metrics `QueryCacheXX` and rename to `QueryPlanCacheXX` [#16289](https://github.com/vitessio/vitess/pull/16289) #### schema management * `schemadiff`/Online DDL internal refactor [#16767](https://github.com/vitessio/vitess/pull/16767) #### vtctldclient @@ -400,6 +406,7 @@ * feat: add a LIMIT 1 on EXISTS subqueries to limit network overhead [#16153](https://github.com/vitessio/vitess/pull/16153) * bugfix: Allow cross-keyspace joins [#16520](https://github.com/vitessio/vitess/pull/16520) * [release-21.0] [Direct PR] fix: route engine to handle column truncation for execute after lookup (#16981) [#16986](https://github.com/vitessio/vitess/pull/16986) + * [release-21.0] Add support for `MultiEqual` opcode for lookup vindexes. (#16975) [#17041](https://github.com/vitessio/vitess/pull/17041) ### Release #### Build/CI * [release-21.0] Fix the release workflow (#16964) [#17020](https://github.com/vitessio/vitess/pull/17020) @@ -416,6 +423,9 @@ * [release-21.0] Code Freeze for `v21.0.0-RC1` [#16912](https://github.com/vitessio/vitess/pull/16912) * [release-21.0] Release of `v21.0.0-RC1` [#16950](https://github.com/vitessio/vitess/pull/16950) * [release-21.0] Bump to `v21.0.0-SNAPSHOT` after the `v21.0.0-RC1` release [#16955](https://github.com/vitessio/vitess/pull/16955) + * [release-21.0] Release of `v21.0.0-RC2` [#17022](https://github.com/vitessio/vitess/pull/17022) + * [release-21.0] Bump to `v21.0.0-SNAPSHOT` after the `v21.0.0-RC2` release [#17049](https://github.com/vitessio/vitess/pull/17049) + * [release-21.0] Add release notes for known issue in v21.0.0 [#17067](https://github.com/vitessio/vitess/pull/17067) ### Testing #### Build/CI * Online DDL flow CI: Update golang version to 1.22.4 [#16066](https://github.com/vitessio/vitess/pull/16066) @@ -446,7 +456,8 @@ * Add a test to verify we respect the overall query timeout [#16800](https://github.com/vitessio/vitess/pull/16800) * flaky test fix for query timeout change [#16821](https://github.com/vitessio/vitess/pull/16821) * Fix error message check on query timeout [#16827](https://github.com/vitessio/vitess/pull/16827) - * upgrade vitess-tester to latest version [#16884](https://github.com/vitessio/vitess/pull/16884) + * upgrade vitess-tester to latest version [#16884](https://github.com/vitessio/vitess/pull/16884) + * Relax vexplain test for upcoming changes [#17035](https://github.com/vitessio/vitess/pull/17035) #### Throttler * Throttler flaky test: explicitly disabling throttler so as to ensure it does not re-enable [#16369](https://github.com/vitessio/vitess/pull/16369) #### VReplication diff --git a/changelog/21.0/21.0.0/release_notes.md b/changelog/21.0/21.0.0/release_notes.md index a97adc5ad64..956d5c3ee1c 100644 --- a/changelog/21.0/21.0.0/release_notes.md +++ b/changelog/21.0/21.0.0/release_notes.md @@ -3,12 +3,15 @@ ### Table of Contents +- **[Known Issue](#known-issues)** + - **[Backup reports itself as successful despite failures](#backup-reports-as-successful)** - **[Major Changes](#major-changes)** - **[Deprecations and Deletions](#deprecations-and-deletions)** - [Deprecated VTTablet Flags](#vttablet-flags) - [Deletion of deprecated metrics](#metric-deletion) - [Deprecated Metrics](#deprecations-metrics) - **[Traffic Mirroring](#traffic-mirroring)** + - **[Atomic Distributed Transaction Support](#atomic-transaction)** - **[New VTGate Shutdown Behavior](#new-vtgate-shutdown-behavior)** - **[Tablet Throttler: Multi-Metric support](#tablet-throttler)** - **[Allow Cross Cell Promotion in PRS](#allow-cross-cell)** @@ -19,13 +22,20 @@ - **[Dynamic VReplication Configuration](#dynamic-vreplication-configuration)** - **[Reference Table Materialization](#reference-table-materialization)** - **[New VEXPLAIN Modes: TRACE and KEYS](#new-vexplain-modes)** - - **[Errant GTID Detection on VTTablets](#errant-gtid-vttablet)** - **[Automatically Replace MySQL auto_increment Clauses with Vitess Sequences](#auto-replace-mysql-autoinc-with-seq)** - **[Experimental MySQL 8.4 support](#experimental-mysql-84)** - **[Current Errant GTIDs Count Metric](#errant-gtid-metric)** - **[vtctldclient ChangeTabletTags](#vtctldclient-changetablettags)** - **[Support for specifying expected primary in reparents](#reparents-expectedprimary)** +## Known Issue + +### Backup reports itself as successful despite failures + +In this release, we identified an issue where a backup may succeed even if a file fails to be backed up. +Leading to a successful backup, even if some errors occurred. +This only happen with the Builtin Backup Engine, and when all files have already been initiated in the backup process. +For more details, please refer to the related GitHub Issue https://github.com/vitessio/vitess/issues/17063. ## Major Changes @@ -81,6 +91,17 @@ $ vtctldclient --server :15999 MoveTables --target-keyspace customer --workflow Mirror rules can be inspected with `GetMirrorRules`. +### Atomic Distributed Transaction Support + +We have introduced atomic distributed transactions as an experimental feature. +Users can now run multi-shard transactions with stronger guarantees. +Vitess now provides two modes of transactional guarantees for multi-shard transactions: Best Effort and Atomic. +These can be selected based on the user’s requirements and the trade-offs they are willing to make. + +Follow the documentation to enable [Atomic Distributed Transaction](https://vitess.io/docs/21.0/reference/features/distributed-transaction/) + +For more details on the implementation and trade-offs, please refer to the [RFC](https://github.com/vitessio/vitess/issues/16245) + ### New VTGate Shutdown Behavior We added a new option to VTGate to disallow new connections while VTGate is shutting down, @@ -209,14 +230,6 @@ filter columns (potential candidates for indexes, primary keys, or sharding keys These new `VEXPLAIN` modes enhance Vitess's query analysis capabilities, allowing for more informed decisions about sharding strategies and query optimization. -### Errant GTID Detection on VTTablets - -VTTablets now run an errant GTID detection logic before they join the replication stream. So, if a replica has an errant GTID, it will -not start replicating from the primary. This protects us from running into situations which are very difficult to recover from. - -For users running with the vitess-operator on Kubernetes, this change means that replica tablets with errant GTIDs will have broken -replication and will report as unready. Users will need to manually replace and clean up these errant replica tablets. - ### Automatically Replace MySQL auto_increment Clauses with Vitess Sequences In https://github.com/vitessio/vitess/pull/16860 we added support for replacing MySQL `auto_increment` clauses with [Vitess Sequences](https://vitess.io/docs/reference/features/vitess-sequences/), performing all of the setup and initialization @@ -244,7 +257,7 @@ The `EmergencyReparentShard` and `PlannedReparentShard` commands and RPCs now su ------------ The entire changelog for this release can be found [here](https://github.com/vitessio/vitess/blob/main/changelog/21.0/21.0.0/changelog.md). -The release includes 354 merged Pull Requests. +The release includes 364 merged Pull Requests. Thanks to all our contributors: @GrahamCampbell, @GuptaManan100, @Utkar5hM, @anshikavashistha, @app/dependabot, @app/vitess-bot, @arthurschreiber, @beingnoble03, @brendar, @cameronmccord2, @chrism1001, @cuishuang, @dbussink, @deepthi, @demmer, @frouioui, @harshit-gangal, @harshitasao, @icyflame, @kirtanchandak, @mattlord, @mattrobenolt, @maxenglander, @mcrauwel, @notfelineit, @perminov, @rafer, @rohit-nayak-ps, @runewake2, @rvrangel, @shanth96, @shlomi-noach, @systay, @timvaillancourt, @vitess-bot diff --git a/changelog/21.0/21.0.0/summary.md b/changelog/21.0/21.0.0/summary.md index 9562c127952..fcac56180a9 100644 --- a/changelog/21.0/21.0.0/summary.md +++ b/changelog/21.0/21.0.0/summary.md @@ -2,12 +2,15 @@ ### Table of Contents +- **[Known Issue](#known-issues)** + - **[Backup reports itself as successful despite failures](#backup-reports-as-successful)** - **[Major Changes](#major-changes)** - **[Deprecations and Deletions](#deprecations-and-deletions)** - [Deprecated VTTablet Flags](#vttablet-flags) - [Deletion of deprecated metrics](#metric-deletion) - [Deprecated Metrics](#deprecations-metrics) - **[Traffic Mirroring](#traffic-mirroring)** + - **[Atomic Distributed Transaction Support](#atomic-transaction)** - **[New VTGate Shutdown Behavior](#new-vtgate-shutdown-behavior)** - **[Tablet Throttler: Multi-Metric support](#tablet-throttler)** - **[Allow Cross Cell Promotion in PRS](#allow-cross-cell)** @@ -24,6 +27,14 @@ - **[vtctldclient ChangeTabletTags](#vtctldclient-changetablettags)** - **[Support for specifying expected primary in reparents](#reparents-expectedprimary)** +## Known Issue + +### Backup reports itself as successful despite failures + +In this release, we identified an issue where a backup may succeed even if a file fails to be backed up. +Leading to a successful backup, even if some errors occurred. +This only happen with the Builtin Backup Engine, and when all files have already been initiated in the backup process. +For more details, please refer to the related GitHub Issue https://github.com/vitessio/vitess/issues/17063. ## Major Changes @@ -79,6 +90,17 @@ $ vtctldclient --server :15999 MoveTables --target-keyspace customer --workflow Mirror rules can be inspected with `GetMirrorRules`. +### Atomic Distributed Transaction Support + +We have introduced atomic distributed transactions as an experimental feature. +Users can now run multi-shard transactions with stronger guarantees. +Vitess now provides two modes of transactional guarantees for multi-shard transactions: Best Effort and Atomic. +These can be selected based on the user’s requirements and the trade-offs they are willing to make. + +Follow the documentation to enable [Atomic Distributed Transaction](https://vitess.io/docs/21.0/reference/features/distributed-transaction/) + +For more details on the implementation and trade-offs, please refer to the [RFC](https://github.com/vitessio/vitess/issues/16245) + ### New VTGate Shutdown Behavior We added a new option to VTGate to disallow new connections while VTGate is shutting down,