Allow backup on tablet master

[guidoiaquinti]

Opening this issue as per conversation in Slack.

Right now there isn't a workflow to recover from a scenario where:

• there's only an active master in a shard (no active replicas)
• there are no backups available for the shard and/or there are only backups "old enough" that master has purged binary logs containing GTIDs that the slave requires.

Afaik we can't trigger a backup on the master using Backup (as it's a master) and we can't change type MASTER -> REPLICA using ChangeSlaveType.

The only workaround that I see is to manually perform a backup on the master, rsyncing the data on the slave and restore replication.

[sougou]

Allowing this is generally useful. It will allow people to deploy vitess on partial clusters where the master need not be served by vitess.

[sougou]

@dweitzman suggested this workaround on Slack:

If you’re in a bind and just want to get another replica set up without losing production data, you use mysqldump to manually make an online logical backup. I think it’s something like mysqldump --all-databases --master-data --single-transaction
Vitess won’t know anything about the logical backup, but you could manually load it on a replica to get yourself back to a world where you have replicas

[dweitzman]

It might be handy if vitess could manage backups in multiple different formats and let you choose how to collect the backups with commands like vtctl Backup -dbtype master -method mysqldump vs vtctl Backup -dbtype rdonly -method drain_copy vs vtctl Backup -dbtype rdonly -method xtrabackup

If we assume that people probably have mysqldump installed (feels like a safe assumption?...) then this could offer an out-of-the-box escape hatch for loss of replicas.

[makmanalp]

With #4699 merged this should be closed now

[deepthi]

Fixed by #4699.
We decided not to allow specifying the backup method via vtctl, it will be continue to be specified at vttablet startup. No support has been added for mysqldump either.