The need to include a mysqlbinlog binary in a k8s deployment

[GenLN]

Overview of the Issue

Here, I just tried using official mysql:8.0.30 image in k8s env

And the reason I did test this is because in Vitess V19.0.0 Release docs says
If you are currently using vitess/lite as your mysqld image in your vitess-operator deployment we invite you to use an official MySQL image, such as mysql:8.0.30
Here is the link

So conclusion is to use vitess/lite image or maybe custom built image from official mysql image as base and add mysqlbinlog

Reproduction Steps

Here I can't take incremental backups only full backup works. Check the output:

commerce/- (zone1-2548885007): time:{seconds:1710231387 nanoseconds:286567256} file:"backup.go" line:152 value:"Starting backup 2024-03-12.081627.zone1-2548885007"
commerce/- (zone1-2548885007): time:{seconds:1710231387 nanoseconds:286628665} file:"builtinbackupengine.go" line:209 value:"Executing Backup at 2024-03-12 08:16:27.28599096 +0000 UTC m=+407.952151989 for keyspace/shard commerce/- on tablet zone1-2548885007, concurrency: 4, compress: true, incrementalFromPos: auto"
commerce/- (zone1-2548885007): time:{seconds:1710231387 nanoseconds:287800266} file:"builtinbackupengine.go" line:256 value:"auto evaluating incremental_from_pos"
commerce/- (zone1-2548885007): time:{seconds:1710231387 nanoseconds:366923115} file:"builtinbackupengine.go" line:263 value:"auto evaluated incremental_from_pos: MySQL56/e2dbb229-e047-11ee-9a75-7a4408213317:1-26"
commerce/- (zone1-2548885007): time:{seconds:1710231387 nanoseconds:405806202} level:ERROR file:"backup.go" line:178 value:"backup is not usable, aborting it: [rpc error: code = Unknown desc = mysqlbinlog not found in any of /usr/{sbin,bin,libexec,scripts}\nreading timestamps from binlog files [vt-2548885007-bin.000002]]"
E0312 09:16:27.471347   38542 main.go:56] rpc error: code = Unknown desc = TabletManager.Backup on zone1-2548885007: reading timestamps from binlog files [vt-2548885007-bin.000002]: rpc error: code = Unknown desc = mysqlbinlog not found in any of /usr/{sbin,bin,libexec,scripts}

And as you can see the official mysql:8.0.30 doesn't have mysqlbinlog in path

bash-4.4$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
bash-4.4$ whereis mysqlbinlog
bash: whereis: command not found
bash-4.4$ mysql
mysql                          mysql_migrate_keyring          mysql_upgrade                  mysqldump
mysql-secret-store-login-path  mysql_ssl_rsa_setup            mysqladmin                     mysqlpump
mysql_config                   mysql_tzinfo_to_sql            mysqld                         mysqlsh
bash-4.4$ mysqlbi

Binary Version

vitess/lite:pitr-test is the image I built based on PR #15440 which is proposed solution for PITR


images:
    vtctld: myrepo/vitess/lite:pitr-test
    vtadmin: vitess/vtadmin:v19.0.0
    vtgate: myrepo/vitess/lite:pitr-test
    vttablet: myrepo/vitess/lite:pitr-test
    vtbackup: myrepo/vitess/lite:pitr-test
    vtorc: myrepo/vitess/lite:pitr-test
    mysqld:
      mysql80Compatible: mysql:8.0.30
    mysqldExporter: prom/mysqld-exporter:v0.14.0

### Operating System and Environment details

```sh
K8S

Log Fragments

No response

[shlomi-noach]

Thank you @GenLN. More context:

This is a followup to #14765. For the purpose of point in time recoveries, the mysqlbinlog binary is required.

I believe this is a documentation change. Modify the changelog to explicitly call out mysqlbinlog.

[mattlord]

Confirmed:

❯ docker run --entrypoint /bin/bash -it mysql:8.0.36 -- command -v mysql
/usr/bin/mysql
❯ docker run --entrypoint /bin/bash -it mysql:8.0.36 -- command -v mysqlbinlog
❯ docker run --entrypoint /bin/bash -it mysql:8.0.36 -- command -v mysqld
/usr/sbin/mysqld

/cc @frouioui as this may complicate our plans to remove MySQL from the vitess/lite images.

[shlomi-noach]

Yeah, the binaries in that image are:

mysql
mysql-secret-store-login-path
mysql_config
mysql_migrate_keyring
mysql_ssl_rsa_setup
mysql_tzinfo_to_sql
mysql_upgrade
mysqladmin
mysqldump
mysqlpump
mysqlsh

[deepthi]

Thank you @GenLN. More context:

This is a followup to #14765. For the purpose of point in time recoveries, the mysqlbinlog binary is required.

I believe this is a documentation change. Modify the changelog to explicitly call out mysqlbinlog.

Beyond that we will need to package mysqlbinlog in our published docker images for k8s users. But as @mattlord pointed out, this complicates things. Which version of mysqlbinlog do we bundle into the docker image?

[frouioui]

I have proposed a fix in #15775 which adds mysqlbinlog back to the lite image while avoiding to re-install all of mysql in the final image.