Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EndtoEnd test cases in Go migrated from Python #5513

Merged
merged 43 commits into from
Dec 10, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
92340d4
Enables vreplication to run directly from MySQL
rafael Oct 10, 2019
3b99f5b
Adds the core of vtshovel program
rafael Oct 11, 2019
f94dd1f
Simplifies vtshovel logic. It assumes that it runs directly again
rafael Oct 16, 2019
1846888
Update approach to not require another binary to run vtshovel
rafael Oct 17, 2019
0c40216
Fixes some bugs in dbconfigs and vstreamer client after inital testing
rafael Oct 18, 2019
064e374
WIP: Adds test for vstreamer client
rafael Oct 30, 2019
eeb8b9c
Adds tests for vstreamer_client
rafael Nov 6, 2019
e8ba26e
Do not pass source conn params around
rafael Nov 6, 2019
e92c6a4
Style improvements
rafael Nov 6, 2019
b4d39e3
Fixes per rebase with file:pos feature
rafael Nov 25, 2019
640b728
Remove unused cruft
morgo Nov 28, 2019
5de685a
vreplication: keyspace_id() support
sougou Aug 5, 2019
0b3a4e9
Rebase test_master with latest Cluster code (#22)
arindamnayak Oct 31, 2019
abc341c
Tabletmanager2 test cases in GO using cluster (#23)
ajeetj Nov 5, 2019
a08ddd5
Vtctld Test cases in Go using cluster (#25)
arindamnayak Nov 5, 2019
febe0fe
Converted schema.py testcase (#26)
arindamnayak Nov 12, 2019
3e3c596
Tablet Manager test cases in Go using cluster (#27)
ajeetj Nov 12, 2019
d312010
Converted keyspace test to go (#28)
arindamnayak Nov 14, 2019
1f09378
Converted sharded test from py to go (#29)
arindamnayak Nov 18, 2019
7c5b3fc
Added testcase for mysqlctl process
saurabh408 Nov 21, 2019
50608ba
removed unused variables
saurabh408 Nov 21, 2019
ead0978
removing comment
saurabh408 Nov 22, 2019
92ff978
addressed review comments
saurabh408 Nov 25, 2019
7d815d8
removed cleandir flag from vttablet teardown
arindamnayak Nov 25, 2019
038c480
vtgate buffer testcase migrated in go
saurabh408 Nov 27, 2019
81e03e8
TabletManager remaining test cases
ajeetj Dec 5, 2019
5968626
moved sql start to non-blocking mode (#35)
arindamnayak Dec 5, 2019
5306db9
added testcase for vtgate vars
arindamnayak Dec 5, 2019
5a2a8f1
redirect process errors to log files (#38)
arindamnayak Dec 5, 2019
153b6c5
debug the test error in ci
arindamnayak Dec 6, 2019
a8c521b
Tabletmanager2 test cases in GO using cluster (#23)
ajeetj Nov 5, 2019
96cd10c
Vtctld Test cases in Go using cluster (#25)
arindamnayak Nov 5, 2019
bbc03d2
Converted schema.py testcase (#26)
arindamnayak Nov 12, 2019
7e93e21
Tablet Manager test cases in Go using cluster (#27)
ajeetj Nov 12, 2019
5858f65
Converted keyspace test to go (#28)
arindamnayak Nov 14, 2019
6a5b1d2
Converted sharded test from py to go (#29)
arindamnayak Nov 18, 2019
8590a22
Added testcase for mysqlctl process
saurabh408 Nov 21, 2019
0def26f
removed unused variables
saurabh408 Nov 21, 2019
1cc8379
removed cleandir flag from vttablet teardown
arindamnayak Nov 25, 2019
afa32c6
TabletManager remaining test cases
ajeetj Dec 5, 2019
f09f3be
moved sql start to non-blocking mode (#35)
arindamnayak Dec 5, 2019
8d20666
Ignoring Python tests which are moved to GO (#40)
ajeetj Dec 9, 2019
0edbf69
resolving merge conflict
arindamnayak Dec 10, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docker/k8s/vtctlclient/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Copyright 2019 The Vitess Authors.
#
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#
# http://www.apache.org/licenses/LICENSE-2.0
#
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
Expand All @@ -28,5 +28,5 @@ COPY --from=k8s /vt/bin/vtctlclient /usr/bin/
# add vitess user/group and add permissions
RUN groupadd -r --gid 2000 vitess && \
useradd -r -g vitess --uid 1000 vitess

CMD ["/usr/bin/vtctlclient"]
6 changes: 3 additions & 3 deletions examples/helm/kvtctld.sh
100755 → 100644
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
#!/bin/bash

# Copyright 2019 The Vitess Authors.
#
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#
# http://www.apache.org/licenses/LICENSE-2.0
#
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
Expand Down
6 changes: 3 additions & 3 deletions examples/kubernetes/etcd-down.sh
100755 → 100644
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
#!/bin/bash

# Copyright 2019 The Vitess Authors.
#
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#
# http://www.apache.org/licenses/LICENSE-2.0
#
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
Expand Down
50 changes: 50 additions & 0 deletions go/test/endtoend/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
This document describe the testing strategy we use for all Vitess components, and the progression in scope / complexity.

As Vitess developers, our goal is to have great end to end test coverage. In the past, these tests were mostly written in python 2.7 is coming to end of life we are moving all of those into GO.


## End to End Tests

These tests are meant to test end-to-end behaviors of the Vitess ecosystem, and complement the unit tests. For instance, we test each RPC interaction independently (client to vtgate, vtgate to vttablet, vttablet to MySQL, see previous sections). But is also good to have an end-to-end test that validates everything works together.

These tests almost always launch a topology service, a few mysqld instances, a few vttablets, a vtctld process, a few vtgates, ... They use the real production processes, and real replication. This setup is mandatory for properly testing re-sharding, cluster operations, ... They all however run on the same machine, so they might be limited by the environment.


## Strategy

All the end to end test are placed under path go/test/endtoend.
The main purpose of grouping them together is to make sure we have single place for reference and to combine similar test to run them in the same cluster and save test running time.

### Setup
All the tests should be launching a real cluster just like the production setup and execute the tests on that setup followed by a teardown of all the services.

The cluster launch functions are provided under go/test/endtoend/cluster. This is still work in progress so feel free to add new function as required or update the existing ones.

In general the cluster is build in following order
- Define Keyspace
- Define Shards
- Start topology service [default etcd]
- Start vtctld client
- Start required mysqld instances
- Start corresponding vttablets (atleast 1 master and 1 replica)
- Start Vtgate

A good example to refer will be go/test/endtoend/clustertest

## Progress
So far we have converted the following Python end to end test cases
- Keyspace tests
- mysqlctl tests
- sharded tests
- tabletmanager tests
- vtgate v3 tests

### In-progress
- Inital sharding
- resharding
- vsplit


After a Python test is migrated in Go it will be removed from end to end ci test run by updating the shard value to 5 in `test/config.json`


Loading