1.45.1

Merged PRs

dolt

• 8699: [sqle] Fix diff table merge join bugs
  Inappropriately using kv merge join, in several ways. No diff table support both for kvexec and diff table indexes aren't sorted, so default merge also fails. Test suite was also being skipped.
  GMS side here: dolthub/go-mysql-server#2803
  fixes: #8700
• 8698: Replace cespare/xxhash with cespare/xxhash/v2
  Currently, we are using two versions of the same package for xxHash:
  https://github.com/dolthub/dolt/blob/d98baafd3e8248a9818e21442f4dfbdeffe78ac4/go/go.mod#L56-L57
  github.com/cespare/xxhash/v2 is the latest version, which includes bug fixes and improvements. This PR updates the codebase to replace github.com/cespare/xxhash with github.com/cespare/xxhash/v2.
  No breaking changes, see https://go.dev/play/p/ZXuwERoBlEi.
• 8691: cache charset bump
• 8684: [stats] stats table name sensitivity tests
  Fix bugs related to table casing, loading deleted tables, and making sure we're using the appropriate branch root when updating statistics.

go-mysql-server

• 2803: [memo] merge joins must use globally sorted indexes
• 2802: exempt processlist column renaming through aliases
  needed for dolphie to work; extension of dolthub/go-mysql-server#2764
• 2799: pool wire write buffer
  BytesBuffer is a class that lets us avoid most allocations for spooling values to wire. Notably, the object is responsible for doubling the backing array size when appropriate, and a Grow(n int) interface is necessary to track when this should happen. Letting the runtime do all of this would be preferable, but the runtime doubles based on slice size, and the refactors required to make that workable are more invasive. We pay for 2 mallocs on doubling, because the first one is never big enough. Not calling Grow after allocing, or growing by too small of length compared to the allocations used will stomp previously written memory.
  As long as we track bytes used with the Grow interface this works smoothly and shaves ~30% off of tablescans.
  perf here: #8693
• 2798: cache session charset
  perf: #8691
• 2796: apply table projections through Distinct nodes
  We weren't pruning table columns when there was a distinct clause over the projections, this resulted the deserialization of every column, even if they weren't going to make it to the result. This is bad for performance, especially if the unread columns are of TEXT, LONGTEXT, 'BLOB, LONGBLOB` type as those are stored out of band, and take longer to deserialize.
  fixes: #8689
• 2795: allow using function as table function

vitess

• 390: Minor bug fixes for caching_sha2_password auth logic
  For accounts without passwords, we need to account for the client sending the null byte when the server re-requests the client auth data, and then skip the AuthMoreDataPacket, and CachingSha2FastAuth packets. Otherwise the mysql client errors with "Malformed packet".
  handleConnectionError is used to report stats about failed connection attempts, but wasn't being called in the correct spot. The previous spot was over counting failed connection attempts, since it was called as part of the auth renegotiation flow. It has been moved to be called whenever we return an error or exit the function without a successful connection.

Closed Issues

• 8700: Panic in join against diff table
• 8689: Prune columns from select distinct
• 8688: P
• 2790: Any plan to make a new patch release?

1.45.0

Backwards incompatible change in this release:

• This release has a small behavior change to the dolt_diff_$table results. Previously changes to the schema of the table, in particular primary key changes, resulted in only the history of the table which was related to the most recent schema. Now the dolt_diff_$table system table will make a best effort to include more history for the table even if we can't perfectly map schema changes.

Per Dolt’s versioning policy, this is a minor version bump because these changes may impact existing applications. Please reach out to us on GitHub or Discord if you have questions or need help with any of these changes.

Merged PRs

dolt

• 8685: update TableFunction
• 8631: Give a little more information in dolt_diff_* when there is a pk change
  This change makes the dolt_diff_* system table a little more forgiving when schema changes occur that we can kind of map from one commit to the next. In the case of the issue, adding a primary key to a key keyless table. This doesn't work in both directions though - if you can't map the schema, we stop walking history (same as before).
  Minor bump required due to behavior of the dolt_diff_* table changing are a result of this change.
  Fixes: #8625

go-mysql-server

• 2795: allow using function as table function
• 2794: Bump go-icu-regex
  Incorporates the fix from here:
  • dolthub/go-icu-regex#2

Closed Issues

• 8625: dolt_diff_* returns empty set for tables altered to add a PK after creating using CREATE TABLE ... AS SELECT
• 8683: dolt table import does not understand a schema file with a primary key defined separately from the column
• 8665: Panic on dolt_diff_* with generated column

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	1.93	0.62	0.32
groupby_scan	13.46	16.41	1.22
index_join	1.47	2.26	1.54
index_join_scan	1.42	1.47	1.04
index_scan	34.33	46.63	1.36
oltp_point_select	0.18	0.27	1.5
oltp_read_only	3.49	5.37	1.54
select_random_points	0.33	0.6	1.82
select_random_ranges	0.37	0.62	1.68
table_scan	34.33	46.63	1.36
types_table_scan	74.46	123.28	1.66
reads_mean_multiplier			1.37

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.9	6.21	0.7
oltp_insert	4.1	3.07	0.75
oltp_read_write	8.9	11.45	1.29
oltp_update_index	4.18	3.13	0.75
oltp_update_non_index	4.18	3.07	0.73
oltp_write_only	5.77	6.21	1.08
types_delete_insert	8.43	6.55	0.78
writes_mean_multiplier			0.87

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	95.58	40.6	2.35
tpcc_tps_multiplier			2.35

Overall Mean Multiple	1.53

1.44.4

Merged PRs

dolt

• 8681: Bump for go-icu-regex
  Bump for:
  • dolthub/go-icu-regex#2
• 8680: fix: panic on null metrics port
  fixes; #8670

go-mysql-server

• 2794: Bump go-icu-regex
  Incorporates the fix from here:
  • dolthub/go-icu-regex#2
• 2793: Revert byte copying optimization

Closed Issues

• 8670: Setting metrics port to null in config.yaml panics

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	1.93	0.62	0.32
groupby_scan	13.7	16.41	1.2
index_join	1.44	2.3	1.6
index_join_scan	1.42	1.47	1.04
index_scan	34.33	46.63	1.36
oltp_point_select	0.18	0.27	1.5
oltp_read_only	3.49	5.37	1.54
select_random_points	0.34	0.59	1.74
select_random_ranges	0.37	0.62	1.68
table_scan	34.33	46.63	1.36
types_table_scan	75.82	123.28	1.63
reads_mean_multiplier			1.36

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.74	6.21	0.71
oltp_insert	4.03	3.07	0.76
oltp_read_write	8.9	11.45	1.29
oltp_update_index	4.1	3.13	0.76
oltp_update_non_index	4.18	3.07	0.73
oltp_write_only	5.57	6.21	1.11
types_delete_insert	8.28	6.55	0.79
writes_mean_multiplier			0.88

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	97.12	41.19	2.36
tpcc_tps_multiplier			2.36

Overall Mean Multiple	1.53

1.44.3

Merged PRs

dolt

• 8677: parse user vars as interface{} instead of always string
  The variable defined for the YAML Marshaller to write user variables to was defined as a map[string]string, which caused all variables to be read as a string even when they weren't quoted. Changing it to a map[string]interface{} fixes that.
  fixes: #8672
• 8675: chore: use errors.New to replace fmt.Errorf with no parameters
  use errors.New to replace fmt.Errorf with no parameters
• 8674: [stats] persist schema hashes for purging
  We previously would flush stats when an active server detected schema changes, but the same problem affects initializing a server's stats after offline table alters. The prototype here uses tags to persist schema hashes so that we detect differences between stored and present table changes.
  Additional improvements to debug logging and error handling. Fixes a separate bug related to re-initializing database statistics after a drop within the same server lifetime.
  Note: There are interruption points in-between detecting schema changes, deleting the schema tags, and deleting the data off disk. There is also an interruption point between writing a schema tag and writing the associated data. I've tried structuring the orderings so that the worst-case scenario is we have to do repeated stats collection work:
  • stats data is only ever written after an associated schema tag is written
  • we delete the tags before deleting data, and missing tags indicates invalid data
    So an interrupted delete will continue to delete after the process is picked up. And an interrupted write will be missed and have to be recollected.
• 8673: fix panic for virtual columns in dolt_diff_<tbl>
  This PR fixes the panic when attempting to view a diff on a table with virtual generated columns by just skipping over the values.
  Ideally, we'd be able to resolve the generated column values themselves and show the diff, but that is not near at hand.
  Fixes: #8665
• 8669: Add BATS tests to cover caching_sha2_password use with mysql client
• 8667: README update
  Added link to blog post with more info on the removal of mysql_native_password authentication in MySQL-9.0 and how to use Dolt's new support for caching_sha2_password authentication.
• 8650: Enable larger statements for dolt_procedures
  Increase the supported stored procedure length from 1K to 8K.

go-mysql-server

• 2793: Revert byte copying optimization
• 2792: Fix the warning checks in enginetest to be more strict
  Previously if an error was expected but none was produced the test would pass.
• 2791: Properly cast ENUMs to TEXT for CASE and CONVERT statements
  fixes: #8598
• 2787: Enable support for caching_sha2_password auth
  This change enables users configured with the caching_sha2_password auth plugin to authenticate to a running SQL server. The default authentication plugin is still mysql_native_password, but users can opt-in to caching_sha2_password by creating a user and explicitly specifying the auth plugin.
  Note that the caching_sha2_password auth plugin requires running the SQL server with a certificate so that TLS connections can be established.
  Depends on: dolthub/vitess#390
  Related to: #8496

Closed Issues

• 8598: enum conversion type has unexpected results
• 8672: Can't set a numeric session variable using user_session_vars section of config.yaml
• 8592: Make Dolt work with mydumper
• 8665: Panic on dolt_diff_* with generated column
• 8496: Support caching_sha2_password authentication method
• 8642: Getting error for select * from dolt_procedures for repo used in dolt docs

1.44.2

Merged PRs

dolt

• 8655: Removed the performance server config block
  Its only member was 'query_parallelism`, which was unused. Existing config files with this key will still parse.

go-mysql-server

• 2788: Fewer wire allocs in SQL() implementations
  Optimizes SQL() implementations that convert interface values into type-specific byte arrays. Combination of skipping runtime.convT checks, that unnecessarily allocated variables to the heap, redundant byte array copying, and other conversion inefficiencies.
  dolt perf here: #8651

  goos: darwin
  goarch: arm64
  pkg: github.com/dolthub/go-mysql-server/sql/types
  cpu: Apple M3 Pro
  │  before.txt  │             after.txt              │
  │    sec/op    │   sec/op     vs base               │
  NumI64SQL-12       62.04n ± 2%   51.13n ± 1%  -17.59% (p=0.002 n=6)
  Varchar10SQL-12    66.85n ± 1%   31.38n ± 2%  -53.06% (p=0.002 n=6)
  TimespanSQL-12     62.36n ± 0%   40.31n ± 1%  -35.35% (p=0.002 n=6)
  TimestampSQL-12   1960.0n ± 1%   255.0n ± 2%  -86.99% (p=0.002 n=6)
  DatetimeSQL-12    1968.5n ± 0%   269.5n ± 3%  -86.31% (p=0.002 n=6)
  EnumSQL-12        111.85n ± 1%   37.49n ± 1%  -66.48% (p=0.002 n=6)
  SetSQL-12         175.15n ± 0%   63.55n ± 1%  -63.72% (p=0.002 n=6)
  BitSQL-12          41.84n ± 1%   41.74n ± 1%        ~ (p=0.589 n=6)
  DecimalSQL-12      683.8n ± 3%   281.9n ± 1%  -58.77% (p=0.002 n=6)
  NumF64SQL-12      105.15n ± 1%   91.72n ± 0%  -12.77% (p=0.002 n=6)
  geomean            189.2n        80.50n       -57.45%
  │  before.txt  │              after.txt              │
  │     B/op     │    B/op     vs base                 │
  NumI64SQL-12        24.00 ± 0%   16.00 ± 0%  -33.33% (p=0.002 n=6)
  Varchar10SQL-12    40.000 ± 0%   8.000 ± 0%  -80.00% (p=0.002 n=6)
  TimespanSQL-12      24.00 ± 0%   16.00 ± 0%  -33.33% (p=0.002 n=6)
  TimestampSQL-12   1520.00 ± 0%   56.00 ± 0%  -96.32% (p=0.002 n=6)
  DatetimeSQL-12    1520.00 ± 0%   56.00 ± 0%  -96.32% (p=0.002 n=6)
  EnumSQL-12        112.000 ± 0%   8.000 ± 0%  -92.86% (p=0.002 n=6)
  SetSQL-12          136.00 ± 0%   16.00 ± 0%  -88.24% (p=0.002 n=6)
  BitSQL-12           16.00 ± 0%   16.00 ± 0%        ~ (p=1.000 n=6) ¹
  DecimalSQL-12       439.0 ± 0%   228.0 ± 0%  -48.06% (p=0.002 n=6)
  NumF64SQL-12        38.00 ± 0%   31.00 ± 0%  -18.42% (p=0.002 n=6)
  geomean             108.0        24.94       -76.92%
  ¹ all samples are equal
  │ before.txt  │              after.txt              │
  │  allocs/op  │ allocs/op   vs base                 │
  NumI64SQL-12       2.000 ± 0%   1.000 ± 0%  -50.00% (p=0.002 n=6)
  Varchar10SQL-12    2.000 ± 0%   1.000 ± 0%  -50.00% (p=0.002 n=6)
  TimespanSQL-12     2.000 ± 0%   1.000 ± 0%  -50.00% (p=0.002 n=6)
  TimestampSQL-12   42.000 ± 0%   3.000 ± 0%  -92.86% (p=0.002 n=6)
  DatetimeSQL-12    42.000 ± 0%   3.000 ± 0%  -92.86% (p=0.002 n=6)
  EnumSQL-12         3.000 ± 0%   1.000 ± 0%  -66.67% (p=0.002 n=6)
  SetSQL-12          5.000 ± 0%   2.000 ± 0%  -60.00% (p=0.002 n=6)
  BitSQL-12          2.000 ± 0%   2.000 ± 0%        ~ (p=1.000 n=6) ¹
  DecimalSQL-12      22.00 ± 0%   10.00 ± 0%  -54.55% (p=0.002 n=6)
  NumF64SQL-12       3.000 ± 0%   2.000 ± 0%  -33.33% (p=0.002 n=6)
  geomean            5.554        1.931       -65.24%
  ¹ all samples are equal
  

• 2787: Enable support for caching_sha2_password auth
  This change enables users configured with the caching_sha2_password auth plugin to authenticate to a running SQL server. The default authentication plugin is still mysql_native_password, but users can opt-in to caching_sha2_password by creating a user and explicitly specifying the auth plugin.
  Note that the caching_sha2_password auth plugin requires running the SQL server with a certificate so that TLS connections can be established.
  Depends on: dolthub/vitess#390
  Related to: #8496
• 2780: Add support for creating users with the caching_sha2_password auth plugin
  This change enables customers to create users configured to authenticate with the caching_sha2_password auth plugin. The generated authentication string uses the same logic as MySQL's caching_sha2_password auth plugin. Users created with caching_sha2_password can not yet authenticate with a GMS server – the next change in this series will enable that.
  Example usage:

  CREATE USER fred@localhost identified with caching_sha2_password by 'pa$$w0rd';

  Depends on: dolthub/vitess#387
  Related to: #8496

vitess

• 390: Minor bug fixes for caching_sha2_password auth logic
  For accounts without passwords, we need to account for the client sending the null byte when the server re-requests the client auth data, and then skip the AuthMoreDataPacket, and CachingSha2FastAuth packets. Otherwise the mysql client errors with "Malformed packet".
  handleConnectionError is used to report stats about failed connection attempts, but wasn't being called in the correct spot. The previous spot was over counting failed connection attempts, since it was called as part of the auth renegotiation flow. It has been moved to be called whenever we return an error or exit the function without a successful connection.
• 389: syntax support for custom explain plan
  For debugging purposes, we replaced MySQL's EXPLAIN output with our very own.
  Unfortunately, it is not causing problems, so we must move our syntax into its own thing.
• 387: Add support for serializing/deserializing caching_sha2_password auth strings

Closed Issues

• 8649: Write timeout not always enforced

1.44.1

Merged PRs

dolt

• 8653: Bug fix: replication fails when a branch is deleted and tag with same name is created
  Our logic to update refs in a read replica was tracking updated refs by path, so when branch b1 was deleted and tag b1 was added, the code was mixing them up and trying to update the branch with the tag ref.
  The fix is to track replicated refs by ID, instead of by path, to avoid collisions. This also changes the first return parameter of the pullBranches function, but none of the three places that call this function assign that first return param to a variable, so that change doesn't affect anything.
• 8643: Support for schema names in table renames
  Also support for changes in enginetest harness setup in GMS

go-mysql-server

• 2784: implement EXPLAIN and EXPLAIN PLAN
  Moving our current implementation of EXPLAIN to EXPLAIN PLAN, and replace EXPLAIN with a dummy implementation of MySQL's EXPLAIN
  Looks like this now:

  tmp2/main> explain select * from t;
  +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
  | id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra |
  +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
  | 1  | SELECT      | NULL  | NULL       | NULL | NULL          | NULL | NULL    | NULL | NULL | NULL     |       |
  +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
  1 row in set (0.00 sec)
  tmp2/main> explain plan select * from t;
  +------------------+
  | plan             |
  +------------------+
  | Table            |
  |  ├─ name: t      |
  |  └─ columns: [i] |
  +------------------+
  3 rows in set (0.00 sec)

  related: #8592

Closed Issues

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	1.89	0.62	0.3
groupby_scan	13.22	16.71	1.3
index_join	1.47	2.3	1.6
index_join_scan	1.42	1.44	1.0
index_scan	34.33	52.89	1.5
oltp_point_select	0.18	0.27	1.5
oltp_read_only	3.49	5.47	1.6
select_random_points	0.34	0.65	1.9
select_random_ranges	0.37	0.65	1.8
table_scan	34.33	52.89	1.5
types_table_scan	75.82	142.39	1.9
reads_mean_multiplier			1.4

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	9.06	6.21	0.7
oltp_insert	4.1	3.07	0.7
oltp_read_write	9.06	11.87	1.3
oltp_update_index	4.18	3.13	0.7
oltp_update_non_index	4.18	3.07	0.7
oltp_write_only	5.77	6.21	1.1
types_delete_insert	8.43	6.67	0.8
writes_mean_multiplier			0.9

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	96.27	40.66	2.4
tpcc_tps_multiplier			2.4

Overall Mean Multiple	1.57

1.44.0

This release contains two potentially backwards incompatible changes:

1. persistence_behavior was removed from config.yaml and it is always default on.
2. EXPLAIN output was changed to match MySQL. Past explain behavior was migrated to EXPLAIN PLAN.

Merged PRs

dolt

• 8646: Allow importing parquet files with arbitrary root column names.
  Parquet represents schemas as trees with a root node, and columns as paths from the root to a leaf.
  parquet-go, the library Dolt uses to export tables as parquet files, uses parquet_go_root as the default name of the root node, but other names are allowed.
  Prior to this PR, Dolt was assuming that the parquet file always had a root node named parquet_go_root, and would fail if any other name was used. This PR changes the behavior to have Dolt read the name from the file instead.
  I also added a test that imports a simple parquet file that was not created with parquet-go.
• 8645: Removed persitence behavior from config
• 8639: Fix for updated issue 8623
  This is an actual fix for the following issue. Additional information was added, which actually exposes the root issue at hand. It's possible for the db/revision combo to make it to branch control, which expects just the branch name (since the revision is given under "branch"). This adds the proper split, so that we always get the database name.
  • #8623
• 8638: go: doltcore: diff: table_deltas.go: Fix perf regression in dolt_add, dolt_commit.
• 8636: Schema fixes for dolt_status, dolt_merge_status, and dolt_constraint_violations_* tables for doltgres

go-mysql-server

• 2784: implement EXPLAIN and EXPLAIN PLAN
  Moving our current implementation of EXPLAIN to EXPLAIN PLAN, and replace EXPLAIN with a dummy implementation of MySQL's EXPLAIN
  Looks like this now:

  tmp2/main> explain select * from t;
  +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
  | id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra |
  +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
  | 1  | SELECT      | NULL  | NULL       | NULL | NULL          | NULL | NULL    | NULL | NULL | NULL     |       |
  +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
  1 row in set (0.00 sec)
  tmp2/main> explain plan select * from t;
  +------------------+
  | plan             |
  +------------------+
  | Table            |
  |  ├─ name: t      |
  |  └─ columns: [i] |
  +------------------+
  3 rows in set (0.00 sec)

  related: #8592
• 2782: avoid fmt.Sprintf and string alloc for time.Sql
  perf here #8640
• 2781: return ok result for select into statements
  Our SELECT ... INTO ... statements return empty result set, which produces strange output in the dolt sql shell.
  MySQL just returns ok results, so we should too.
  discovered in: dolthub/go-mysql-server#2779
• 2779: fix AS OF clause panic for certain expressions
  We attempt to parse eval AS OF expressions in the builder (because we assume it is going to be a literal), but Subqueries cannot be evaluated until after they have gone through the analyzer.
  partially addresses: #8635
• 2776: bump mysql version
  Certain tools expect a higher version of MySQL.
  Currently, the latest stable version of MySQL is 8.4.4, but 8.0.23 is the minimum needed to satisfy mydumper.
  Additionally, this alters the version() method to select directly from the @@version system variable.
  related: #8592

vitess

• 389: syntax support for custom explain plan
  For debugging purposes, we replaced MySQL's EXPLAIN output with our very own.
  Unfortunately, it is not causing problems, so we must move our syntax into its own thing.
• 386: parse io_threads and sql_state as no-ops
  related: #8592
• 384: add mariadb table opts and tests
  This PR adds support for parsing the mariadb table options listed here:
  https://mariadb.com/kb/en/create-table/#table-options
  They are all no-ops (much like many of the MySQL table options)
  related: #8592

Closed Issues

• 8623: Branch-qualified DSN gets different branch permissions

1.43.20

Merged PRs

dolt

• 8634: Add pointers for schemas for dolt_workspace_* and dolt_diff system tables for doltgres
• 8633: Add TLS options in help docs
• 8632: fix error message for duplicate unique key violation on out of band types
  Certain types are stored out of band (like TEXT and BLOB), so we compare their addresses to test for existence. Consequently, the error message would contain the hash rather than the contents.
  This PR fixes the formatting to print the prefix.
  fixes: #8629
• 8630: Fixed issues with branch control
  This is primarily to fix issues with:
  • #8622
  • #8623
    For the first issue, we did not actually update the longest match counter, and it was therefore being ignored. The variable existed, but it was not being updated. I think this went so long without being caught (even with all of the tests in enginetests and bats) due to how we handle expression folding, where subset entries are completely ignored if a superset already exists.
    The second issue is somewhat similar to the first, in that we were not checking the permission set when determining whether a new row is actually a subset. If an existing row is a superset, but the permissions are more restrictive than the new row, then it's not an actual superset, since it has more restrictive permissions. This is even in the documentation, so it's a simple mistake of forgetting to add that check to the code itself.
    Lastly, the bats tests have been updated such that they rely on sockets now, and Windows does not support sockets. This was not caught since most Windows developers have moved to primary development on Doltgres, so this at least allows such developers to run the bats tests locally again.
• 8624: Update sql status to not report changes for unstaged tables that have changed but not visibly.
  Sometimes tables can change their hashes despite having no visible changes. The two ways I identified where this can happen are:
  • The table has different column tags from the ancestor because it was originally a different table that got renamed.
  • The table stores whether it's in a conflict and stores the hash of the other branch's table and the common ancestor table.
    If these are the only changes to a table, we shouldn't report it as changed in dolt status.
    These tables can still be staged and report as modified once staged.
• 8619: README updates to explain MySQL 8.4 requirement
  Updates README to explain how to install mysql 8.4, since the 9.0 innovation release does not include support for the mysql_native_password auth plugin by default.

go-mysql-server

• 2776: bump mysql version
  Certain tools expect a higher version of MySQL.
  Currently, the latest stable version of MySQL is 8.4.4, but 8.0.23 is the minimum needed to satisfy mydumper.
  Additionally, this alters the version() method to select directly from the @@version system variable.
  related: #8592
• 2775: prevent creating and dropping mysql and information_schema databases
  fixes: #8621
• 2774: insert ignore to enum column truncates data
• 2765: increase column size for information_schema.processlist.state
  The State column in our information_schema.processlist table is different than the MySQL implementation because we include progress information. As a result, it is possible to exceed the 64 character limit that the column is defined as.
  However, thius means deviating from MySQL's definition of the table.
• 2764: information_schema uppercase rule doesn't apply to processlist table
  Selecting from information schema typically results in all uppercase column names, except for the information_schema.processlist table, which follows the non info schema casing (matches the projection).
  Relevant MySQL Bug: https://bugs.mysql.com/bug.php?id=84456

Closed Issues

• 8629: Erroneous output with unique constraint on TEXT or TINYTEXT
• 8607: Creating two tables with the same schema and database, dropping one of them, and renaming the second table to the dropped table creates an entry in status but not diff
• 8622: Potential issue with longest matching branch namespace control rules
• 8621: prevent CREATE DATABASE mysql;
• 8611: INSERT IGNORE for ENUM should truncate data instead of using the first element

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	1.89	0.62	0.3
groupby_scan	13.22	16.41	1.2
index_join	1.47	2.26	1.5
index_join_scan	1.42	1.47	1.0
index_scan	34.95	55.82	1.6
oltp_point_select	0.18	0.27	1.5
oltp_read_only	3.43	5.47	1.6
select_random_points	0.33	0.65	2.0
select_random_ranges	0.36	0.65	1.8
table_scan	34.95	55.82	1.6
types_table_scan	77.19	144.97	1.9
reads_mean_multiplier			1.5

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.9	6.21	0.7
oltp_insert	4.1	3.07	0.7
oltp_read_write	8.9	11.65	1.3
oltp_update_index	4.18	3.13	0.7
oltp_update_non_index	4.18	3.07	0.7
oltp_write_only	5.67	6.21	1.1
types_delete_insert	8.43	6.55	0.8
writes_mean_multiplier			0.9

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	95.75	41.26	2.3
tpcc_tps_multiplier			2.3

Overall Mean Multiple	1.57

1.43.19

Merged PRs

dolt

• 8612: go/store/nbs: Fixing GCGen to be more correct.
  The original purpose of gc gen was two fold. The first purpose was to avoid applying the garbage collection results if the store had changed due to multi-process concurrency for any reason. The second purpose was to fast-complete a dolt gc invocation if the store had not changed at all since the last GC run.
  For the first purpose, it is no longer necessary. We no longer allow multi-process access to the same NomsBlockStore.
  For the second purpose, it was implemented slightly incorrectly, given the introduction of dolt gc --full. This change fixes the implementation to be more correct.
  In particular, the semantics are:
  • After a dolt gc --full, an immediate invocation of dolt gc or dolt gc --full fast-completes as no collection being necessary.
  • After a dolt gc, only a dolt gc fast-completes as no collection being necessary. A dolt gc --full will run a full GC to completion.

go-mysql-server

• 2765: increase column size for information_schema.processlist.state
  The State column in our information_schema.processlist table is different than the MySQL implementation because we include progress information. As a result, it is possible to exceed the 64 character limit that the column is defined as.
  However, thius means deviating from MySQL's definition of the table.
• 2762: Add schema to StatsQualifier for schema databases

Closed Issues

• 8385: Permissions / branch control incorrect for branch-qualified database names

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	2.11	0.62	0.3
groupby_scan	13.22	16.71	1.3
index_join	1.37	2.26	1.6
index_join_scan	1.27	1.44	1.1
index_scan	34.33	54.83	1.6
oltp_point_select	0.18	0.27	1.5
oltp_read_only	3.49	5.37	1.5
select_random_points	0.34	0.65	1.9
select_random_ranges	0.39	0.65	1.7
table_scan	34.95	55.82	1.6
types_table_scan	74.46	147.61	2.0
reads_mean_multiplier			1.5

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	8.9	6.21	0.7
oltp_insert	4.1	3.07	0.7
oltp_read_write	9.06	11.65	1.3
oltp_update_index	4.18	3.13	0.7
oltp_update_non_index	4.18	3.07	0.7
oltp_write_only	5.67	6.21	1.1
types_delete_insert	8.43	6.55	0.8
writes_mean_multiplier			0.9

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	96.58	40.39	2.4
tpcc_tps_multiplier			2.4

Overall Mean Multiple	1.60

1.43.18

Merged PRs

dolt

• 8600: Add testing for show slave status
  This syntax is deprecated and will be removed in future MySQL versions (SHOW REPLICA STATUS should be preferred instead). However, some tools (e.g. Dolphie, MyDumper) still rely on this deprecated syntax, so we're adding support for it to keep compatibility with those tools.
  Depends on:
  • dolthub/vitess#381
  • dolthub/go-mysql-server#2767
• 8596: go/store/nbs: table_index.go: Improve support for large table files.
  Support suffix arrays larger than 4GB by avoiding uint32 overflows when matching chunk suffixes.
• 8595: replace nil defaults with empty string
  Needed for dolphie.
• 8590: system variables cannot have nil defaults
  related: #8585 (comment)
• 8589: Fix dolt_statistics table for multiple schemas for doltgres
  GMS PR: dolthub/go-mysql-server#2762
• 8561: [kvexec] merge join
  This isn't the best perf win on linux, but it counteracts the sql.Row interface PR which otherwise would swing merge join +30% in the wrong direction.

  goos: darwin
  goarch: arm64
  pkg: github.com/dolthub/dolt/go/performance/microsysbench
  │  before.txt  │           after.txt           │
  │    sec/op    │    sec/op     vs base         │
  OltpJoinScan-12   680.6µ ± 26%   612.1µ ± 17%  ~ (p=0.240 n=6)
  │  before.txt  │              after.txt              │
  │     B/op     │     B/op      vs base               │
  OltpJoinScan-12   163.8Ki ± 0%   123.8Ki ± 0%  -24.42% (p=0.002 n=6)
  │ before.txt  │             after.txt              │
  │  allocs/op  │  allocs/op   vs base               │
  OltpJoinScan-12   5.906k ± 0%   4.233k ± 0%  -28.33% (p=0.002 n=6)
  

  TODO:
  • left join
  • nulls and other edge cases
  • execute full comparer

go-mysql-server

• 2773: Fixed error in setup found by user
• 2769: support NO_AUTO_CREATE_USER option in sql_mode
  Older MySQL 5.7 Docs; https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sqlmode_no_auto_create_user
  We already do this by default, so this is just parsing the option and does nothing.
• 2767: Add support for show slave status
  This syntax is deprecated and will be removed in future MySQL versions (SHOW REPLICA STATUS should be preferred instead). However, some tools (e.g. Dolphie, MyDumper) still rely on this deprecated syntax, so we're adding support for it to keep compatibility with those tools.
  Depends on: dolthub/vitess#381
• 2766: add case for enum return types in multi-branch case statement
  Hotfix for Enum conversion issue:
  #8598
  Doesn't address real issue, which revolves around conversion for CASE.
• 2763: Better error message for invalid charset string
  This is a followup to the comments on this PR:
  • dolthub/go-mysql-server#2758 (comment)
• 2762: Add schema to StatsQualifier for schema databases
• 2761: add innodb_buffer_pool_size system variable
  docs: https://dev.mysql.com/doc/refman/8.4/en/innodb-parameters.html#sysvar_innodb_buffer_pool_size
• 2759: ignore empty strings in sets with multiple values
  fixes: #8584
• 2756: Update README - fix db name when setting context
  I'm pretty sure this needs to match with memory.NewDatabase(dbName) on line 135

vitess

• 384: add mariadb table opts and tests
  This PR adds support for parsing the mariadb table options listed here:
  https://mariadb.com/kb/en/create-table/#table-options
  They are all no-ops (much like many of the MySQL table options)
  related: #8592
• 383: parse start transaction with consistent as no-op
  related: #8592
• 382: Add syntax support to allow CREATE USER statements to quote the auth plugin
  MySQL allows the auth plugin name to be quoted or unquoted, but our parser previously only supported the unquoted mode for most forms of CREATE USER. This change adds support for the following forms:
  • CREATE USER user@localhost IDENTIFIED WITH 'auth_plugin';
  • CREATE USER user@localhost IDENTIFIED WITH 'auth_plugin' BY random password;
  • CREATE USER user@localhost IDENTIFIED WITH 'auth_plugin' BY 'password';
• 381: Add support for show slave status
  This syntax is deprecated and will be removed in future MySQL versions (SHOW REPLICA STATUS should be preferred instead). However, some tools (e.g. Dolphie, MyDumper) still rely on this deprecated syntax, so we're adding support for it to keep compatibility with those tools.
• 379: Refactor AuthServer to support multiple authentication methods
  In preparation of supporting caching_sha2_password authentication, this change applies the refactoring from vitessio/vitess#8503 to our fork of Vitess so that we can more easily support multiple authentication methods.
  Related to: #8496

Closed Issues