Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Downgrade Testing #9300

Merged
merged 21 commits into from
Jan 6, 2022
Merged

Upgrade Downgrade Testing #9300

Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
62a0334
test: addition of the initial code for the upgrade downgrade test wor…
frouioui Nov 30, 2021
98666aa
test: assert the completion of the two builds (other and current vers…
frouioui Nov 30, 2021
7f1332d
test: remove unrequired get dependencies steps
frouioui Nov 30, 2021
2f71e4d
feat: copy other vesion's vtgate binary to bin
frouioui Dec 8, 2021
1bf12cd
feat: start the E2E twice with different versions every time
frouioui Dec 8, 2021
1435b0c
feat: create a vitess cluster with current version binaries
frouioui Dec 9, 2021
efbc3bc
feat: testing a new end to end test package for query serving in the …
frouioui Dec 9, 2021
2006b9a
feat: run the upgrade downgrade test only if the pull request has the…
frouioui Dec 9, 2021
f8b19f9
feat: revert conditional label check to start upgrade/downgrade
frouioui Dec 14, 2021
5efe626
feat: fetch latest version SHA and use it against current SHA
frouioui Dec 14, 2021
d2588f6
feat: start the two main jobs of the workflow only when the upgrade d…
frouioui Dec 14, 2021
2b211e0
fix: installing mysql80 properly with ubuntu-latest
frouioui Dec 16, 2021
da0da16
feat: include more E2E tests in the upgrade downgrade suite
frouioui Jan 4, 2022
a4aa85f
Empty Commit to restart the CI
frouioui Jan 4, 2022
69276be
feat: change the list of included tests for the upgrade downgrade suite
frouioui Jan 4, 2022
f4ba5ec
Delete all.sh
frouioui Jan 4, 2022
eeccfb2
Merge remote-tracking branch 'upstream/main' into upgrade-downgrade
frouioui Jan 5, 2022
903b2c3
feat: change the query serving upgrade downgrade workflow name
frouioui Jan 5, 2022
98d6f42
feat: allow the workflow to be skipped instead of using a label to en…
frouioui Jan 5, 2022
fd9ef15
Empty Commit to restart the CI
frouioui Jan 6, 2022
fdd4af8
feat: change how we check the skip label
frouioui Jan 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
189 changes: 189 additions & 0 deletions .github/workflows/upgrade_downgrade_test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
name: Upgrade Downgrade Testing
frouioui marked this conversation as resolved.
Show resolved Hide resolved
on:
push:
pull_request:

concurrency:
group: format('{0}-{1}', ${{ github.ref }}, 'Upgrade Downgrade Testing')
cancel-in-progress: true

# This test ensures that our end-to-end tests work using Vitess components
# (vtgate, vttablet, etc) built on different versions.

jobs:
get_upgrade_downgrade_label:
if: github.repository == 'vitessio/vitess'
name: Get the Upgrade Downgrade pull request label
runs-on: ubuntu-latest
outputs:
hasLabel: ${{ steps.check_label.outputs.hasLabel }}

steps:
- name: Check Label
uses: Dreamcodeio/pr-has-label-action@master
id: check_label
with:
label: Upgrade Downgrade

get_latest_release:
if: always() && (github.event_name != 'pull_request' || needs.get_upgrade_downgrade_label.outputs.hasLabel == 'true')
name: Get latest release
runs-on: ubuntu-latest
needs:
- get_upgrade_downgrade_label
outputs:
latest_release: ${{ steps.set-outpout.outputs.matrix }}

steps:
- name: Check out to HEAD
uses: actions/checkout@v2
with:
fetch-depth: 0

- name: Get latest major release of Vitess
run: |
last_major_releases=$(git show-ref --tags | grep -E 'refs/tags/v[0-9]*.[0-9]*.0$' | sed 's/[a-z0-9]* refs\/tags\/v//' | sort -nr | head -n1)
echo "latest_releases=$(echo "$last_major_releases" | awk ' BEGIN { ORS = ""; print "["; } { print "\/\@{\\\"project\\\":\\\"v"$0"\\\"}\/\@"; } END { print "]"; }' | sed "s^\/\@\/\@^, ^g;s^\/\@^^g")" >> $GITHUB_ENV

- name: Set output
id: set-outpout
run: |
echo "::set-output name=matrix::{\"include\":${{ env.latest_releases }} }"

upgrade_downgrade_test:
if: always() && (github.event_name != 'pull_request' || needs.get_upgrade_downgrade_label.outputs.hasLabel == 'true')
name: Run Upgrade Downgrade Test
runs-on: ubuntu-latest
needs:
- get_upgrade_downgrade_label
- get_latest_release
strategy:
fail-fast: false
matrix: ${{fromJSON(needs.get_latest_release.outputs.latest_release)}}

steps:
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.17

- name: Set up python
uses: actions/setup-python@v2

- name: Tune the OS
run: |
echo '1024 65535' | sudo tee -a /proc/sys/net/ipv4/ip_local_port_range

- name: Get base dependencies
run: |
sudo DEBIAN_FRONTEND="noninteractive" apt-get update
# Uninstall any previously installed MySQL first
sudo systemctl stop apparmor
sudo DEBIAN_FRONTEND="noninteractive" apt-get remove -y --purge mysql-server mysql-client mysql-common
sudo apt-get -y autoremove
sudo apt-get -y autoclean
sudo deluser mysql
sudo rm -rf /var/lib/mysql
sudo rm -rf /etc/mysql
# Install mysql80
wget -c https://dev.mysql.com/get/mysql-apt-config_0.8.20-1_all.deb
echo mysql-apt-config mysql-apt-config/select-server select mysql-8.0 | sudo debconf-set-selections
sudo DEBIAN_FRONTEND="noninteractive" dpkg -i mysql-apt-config*
sudo apt-get update
sudo DEBIAN_FRONTEND="noninteractive" apt-get install -y mysql-server mysql-client
# Install everything else we need, and configure
sudo apt-get install -y make unzip g++ etcd curl git wget eatmydata
sudo service mysql stop
sudo service etcd stop
sudo bash -c "echo '/usr/sbin/mysqld { }' > /etc/apparmor.d/usr.sbin.mysqld" # https://bugs.launchpad.net/ubuntu/+source/mariadb-10.1/+bug/1806263
sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/
sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld || echo "could not remove mysqld profile"

# install JUnit report formatter
go get -u github.com/vitessio/go-junit-report@HEAD

wget https://repo.percona.com/apt/percona-release_latest.$(lsb_release -sc)_all.deb
sudo apt-get install -y gnupg2
sudo dpkg -i percona-release_latest.$(lsb_release -sc)_all.deb
sudo apt-get update
sudo apt-get install percona-xtrabackup-24

# Checkout to the last release of Vitess
- name: Check out other version's code (${{ matrix.project }})
uses: actions/checkout@v2
with:
ref: ${{ matrix.project }}

- name: Get dependencies for the last release
run: |
go mod download

- name: Building last release's binaries
timeout-minutes: 10
run: |
source build.env
make build
mkdir -p /tmp/vitess-build-other/
cp -R bin /tmp/vitess-build-other/
rm -Rf bin/*

# Checkout to this build's commit
- name: Check out commit's code
uses: actions/checkout@v2

- name: Get dependencies for this commit
run: |
go mod download

- name: Building the binaries for this commit
timeout-minutes: 10
run: |
source build.env
make build
mkdir -p /tmp/vitess-build-current/
cp -R bin /tmp/vitess-build-current/

# Running a test with vtgate and vttablet using version n
- name: Run query serving tests (vtgate=N, vttablet=N)
run: |
rm -rf /tmp/vtdataroot
mkdir -p /tmp/vtdataroot

source build.env
eatmydata -- go run test.go -skip-build -keep-data -docker=false -print-log -follow -tag upgrade_downgrade_query_serving

# Swap the binaries in the bin. Use vtgate version n-1 and keep vttablet at version n
- name: Use last release's VTGate
run: |
source build.env

cp /tmp/vitess-build-other/bin/vtgate $PWD/bin/vtgate
vtgate --version

# Running a test with vtgate at version n-1 and vttablet at version n
- name: Run query serving tests (vtgate=N-1, vttablet=N)
run: |
rm -rf /tmp/vtdataroot
mkdir -p /tmp/vtdataroot

source build.env
eatmydata -- go run test.go -skip-build -keep-data -docker=false -print-log -follow -tag upgrade_downgrade_query_serving

# Swap the binaries again. This time, vtgate will be at version n, and vttablet will be at version n-1
- name: Use current version VTGate, and other version VTTablet
run: |
source build.env

cp /tmp/vitess-build-current/bin/vtgate $PWD/bin/vtgate
cp /tmp/vitess-build-other/bin/vttablet $PWD/bin/vttablet
vtgate --version
vttablet --version

# Running a test with vtgate at version n and vttablet at version n-1
- name: Run query serving tests (vtgate=N, vttablet=N-1)
run: |
rm -rf /tmp/vtdataroot
mkdir -p /tmp/vtdataroot

source build.env
eatmydata -- go run test.go -skip-build -keep-data -docker=false -print-log -follow -tag upgrade_downgrade_query_serving
14 changes: 7 additions & 7 deletions test/config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -676,7 +676,7 @@
"Manual": false,
"Shard": "vtgate_queries",
"RetryMax": 2,
"Tags": []
"Tags": ["upgrade_downgrade_query_serving"]
},
"vtgate_queries_subquery": {
"File": "unused.go",
Expand All @@ -685,7 +685,7 @@
"Manual": false,
"Shard": "vtgate_queries",
"RetryMax": 2,
"Tags": []
"Tags": ["upgrade_downgrade_query_serving"]
},
"vtgate_queries_union": {
"File": "unused.go",
Expand All @@ -694,7 +694,7 @@
"Manual": false,
"Shard": "vtgate_queries",
"RetryMax": 2,
"Tags": []
"Tags": ["upgrade_downgrade_query_serving"]
},
"vtgate_buffer": {
"File": "unused.go",
Expand All @@ -721,7 +721,7 @@
"Manual": false,
"Shard": "vtgate_schema",
"RetryMax": 1,
"Tags": []
"Tags": ["upgrade_downgrade_query_serving"]
},
"vtgate_schematracker_loadkeyspace": {
"File": "unused.go",
Expand All @@ -748,7 +748,7 @@
"Manual": false,
"Shard": "vtgate_schema_tracker",
"RetryMax": 1,
"Tags": []
"Tags": ["upgrade_downgrade_query_serving"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it expected that there will be more tags of this kind?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am expecting to have at least two more, one for backups and one for cluster management

},
"vtgate_schematracker_unsharded": {
"File": "unused.go",
Expand All @@ -757,7 +757,7 @@
"Manual": false,
"Shard": "vtgate_schema_tracker",
"RetryMax": 1,
"Tags": []
"Tags": ["upgrade_downgrade_query_serving"]
},
"vtgate_mysql80": {
"File": "unused.go",
Expand Down Expand Up @@ -865,7 +865,7 @@
"Manual": false,
"Shard": "vtgate_vschema",
"RetryMax": 1,
"Tags": []
"Tags": ["upgrade_downgrade_query_serving"]
},
"vtgate_readafterwrite": {
"File": "unused.go",
Expand Down