Auto-detect VT_MYSQL_ROOT, remove cruft

[morgo]

The changes are:

• Push the logic for VT_MYSQL_ROOT autodetection into mysqlctl, and then remove the auto-detection and setting of this variable from Docker images (PATH works fine! It's predictable behavior..).
• MYSQL_FLAVOR is removed, since it has not been required for some time since auto-detection was implemented based on discovering a local binary, and running --version on it.
• There are also Docker variables for GOTOP/PYTOP, which don't seem to be used. It is a good opportunity to remove some of the ceremony around bootstrapping.
• The old docker images around building the website and packaging are removed.
• I don't believe we require pkg-config or mecurial any more.
• We are not using azure code pipelines.
• GOPATH,GOBIN, GO111MODULES - we can just set this in the Makefile, since there is no reason to overwrite it.

Signed-off-by: Morgan Tocker [email protected]

[morgo]

I am currently running rebuilding docker images and testing locally. I will mark this as ready-for-review after it completes.

[morgo]

It passed. Prepared_statement test failed, but that is an existing problem:

make docker_bootstrap && go run test.go -pull=false 
...
2019/11/28 17:06:03 mysql57.backup                          	FLAKY (1/2 failed)
2019/11/28 17:06:03 mysql57.backup_mysqlctld                	FLAKY (1/2 failed)
2019/11/28 17:06:03 mysql57.backup_only                     	PASS
2019/11/28 17:06:03 mysql57.backup_transform                	PASS
2019/11/28 17:06:03 mysql57.backup_transform_mysqlctld      	PASS
2019/11/28 17:06:03 mysql57.binlog                          	PASS
2019/11/28 17:06:03 mysql57.cache_invalidation              	PASS
2019/11/28 17:06:03 mysql57.cell_aliases                    	PASS
2019/11/28 17:06:03 mysql57.cell_no_aliases                 	PASS
2019/11/28 17:06:03 mysql57.check_make_parser               	PASS
2019/11/28 17:06:03 mysql57.client_test                     	PASS
2019/11/28 17:06:03 mysql57.cluster_endtoend                	PASS
2019/11/28 17:06:03 mysql57.custom_sharding                 	PASS
2019/11/28 17:06:03 mysql57.e2e_race                        	PASS
2019/11/28 17:06:03 mysql57.encrypted_replication           	PASS
2019/11/28 17:06:03 mysql57.encrypted_transport             	PASS
2019/11/28 17:06:03 mysql57.endtoend                        	PASS
2019/11/28 17:06:03 mysql57.initial_sharding                	PASS
2019/11/28 17:06:03 mysql57.initial_sharding_bytes          	PASS
2019/11/28 17:06:03 mysql57.initial_sharding_multi          	PASS
2019/11/28 17:06:03 mysql57.java                            	PASS
2019/11/28 17:06:03 mysql57.keyrange                        	PASS
2019/11/28 17:06:03 mysql57.keyspace                        	PASS
2019/11/28 17:06:03 mysql57.legacy_resharding               	PASS
2019/11/28 17:06:03 mysql57.local_example                   	PASS
2019/11/28 17:06:03 mysql57.merge_sharding                  	PASS
2019/11/28 17:06:03 mysql57.merge_sharding_bytes            	PASS
2019/11/28 17:06:03 mysql57.messaging                       	PASS
2019/11/28 17:06:03 mysql57.mysql_server                    	PASS
2019/11/28 17:06:03 mysql57.mysqlctl                        	PASS
2019/11/28 17:06:03 mysql57.prepared_statement              	FAIL (3 tries)
2019/11/28 17:06:03 mysql57.python_client                   	PASS
2019/11/28 17:06:03 mysql57.recovery                        	PASS
2019/11/28 17:06:03 mysql57.reparent                        	PASS
2019/11/28 17:06:03 mysql57.resharding                      	PASS
2019/11/28 17:06:03 mysql57.resharding_bytes                	PASS
2019/11/28 17:06:03 mysql57.resharding_rbr                  	PASS
2019/11/28 17:06:03 mysql57.schema                          	PASS
2019/11/28 17:06:03 mysql57.sharded                         	PASS
2019/11/28 17:06:03 mysql57.sharded_recovery                	PASS
2019/11/28 17:06:03 mysql57.tabletmanager                   	PASS
2019/11/28 17:06:03 mysql57.tabletmanager_consul            	PASS
2019/11/28 17:06:03 mysql57.tabletmanager_etcd2             	PASS
2019/11/28 17:06:03 mysql57.unit                            	FLAKY (1/2 failed)
2019/11/28 17:06:03 mysql57.unit_race                       	PASS
2019/11/28 17:06:03 mysql57.update_stream                   	PASS
2019/11/28 17:06:03 mysql57.update_stream_rbr               	PASS
2019/11/28 17:06:03 mysql57.vertical_split                  	PASS
2019/11/28 17:06:03 mysql57.vertical_split_rbr              	PASS
2019/11/28 17:06:03 mysql57.vschema                         	PASS
2019/11/28 17:06:03 mysql57.vtctld                          	PASS
2019/11/28 17:06:03 mysql57.vtctld_web                      	PASS
2019/11/28 17:06:03 mysql57.vtgate_buffer                   	PASS
2019/11/28 17:06:03 mysql57.vtgate_utils                    	PASS
2019/11/28 17:06:03 mysql57.vtgatev2                        	PASS
2019/11/28 17:06:03 mysql57.vtgatev3                        	PASS
2019/11/28 17:06:03 mysql57.vttest_sample                   	PASS
2019/11/28 17:06:03 mysql57.worker                          	PASS
2019/11/28 17:06:03 mysql57.xb_recovery                     	PASS
2019/11/28 17:06:03 mysql57.xtrabackup                      	PASS
2019/11/28 17:06:03 mysql57.xtrabackup_xbstream             	PASS
2019/11/28 17:06:03 ============================================================
2019/11/28 17:06:03 57 PASSED, 3 FLAKY, 1 FAILED, 0 SKIPPED
2019/11/28 17:06:03 Total time: 2h24m37.8s

[morgo]

@sougou There is a race in the VReplication testsuite, that I can reproduce about 20% of the time:

ok  	vitess.io/vitess/go/vt/vttablet/tabletmanager	0.064s
E1129 03:19:34.913983   17312 binlog_player.go:202] error (expected error) in selecting vreplication settings select pos, stop_pos, max_tps, max_replication_lag, state from _vt.vreplication where id=1
E1129 03:19:34.914299   17312 controller.go:144] stream 1: error (expected error) in selecting vreplication settings select pos, stop_pos, max_tps, max_replication_lag, state from _vt.vreplication where id=1, retrying after 10ms
E1129 03:19:35.240267   17312 engine.go:596] Error waiting for pos: MariaDB/0-1-1084, last pos: MariaDB/0-1-1083: context canceled, wait time: 10.4µs
--- FAIL: TestPlayerCopyTableContinuation (3.86s)
##[error]    vcopier_test.go:517: query:
        "update _vt.vreplication set pos='MySQL56/0f4697bc-1257-11ea-a874-000d3a0d951e:1-160', time_updated=1574997581, transaction_timestamp=1574997579, message='' where id=13", does not match query 0:
        "/update _vt.vreplication set state='Running'"
E1129 03:19:44.356040   17312 controller.go:144] stream 25: Duplicate column name 'val2' (errno 1060) (sqlstate 42S21) during query: alter table t1 add column val2 varchar(128), retrying after 5s
E1129 03:19:51.131790   17312 controller.go:144] stream 35: vstream ended, retrying after 1ms
FAIL
FAIL	vitess.io/vitess/go/vt/vttablet/tabletmanager/vreplication	22.294s

I will leave this test as failed so you can inspect. But if you hit the "Re-run checks" button, I guarantee unit will pass :-)

[morgo]

It also looks like the unit_race test has been finding races, and they are incorrectly not being reported.

[dkhenry]

I think @gedgar still uses the packaging to make RPM files

[sougou]

--- FAIL: TestPlayerCopyTableContinuation (3.86s)
##[error] vcopier_test.go:517: query:
"update _vt.vreplication set pos='MySQL56/0f4697bc-1257-11ea-a874-000d3a0d951e:1-160', time_updated=1574997581, transaction_timestamp=1574997579, message='' where id=13", does not match query 0:
"/update _vt.vreplication set state='Running'"

I checked this (and have seen this before). It's hard to avoid. It happens when things become so slow (>1s), that the streamer ends up sending a heartbeat event. We can live with this for now. We can later parameterize the heartbeat interval to a bigger value during tests.