Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

EndtoEnd test cases in Go migrated from Python #5513

Merged
merged 43 commits into from
Dec 10, 2019
Merged

EndtoEnd test cases in Go migrated from Python #5513

merged 43 commits into from
Dec 10, 2019

Conversation

ajeetj
Copy link
Contributor

@ajeetj ajeetj commented Dec 5, 2019 •
edited
Loading

Below are the list of tests that we have migrated so far.

  • Keyspace tests
  • mysqlctl tests
  • sharded tests
  • tabletmanager tests
    • commands
    • custom rule
    • lock unlock
    • qps
    • health check
    • security policy
  • vtgate tests
    • buffer
    • schema
    • sequence
    • transaction
    • vschema
  • improvement in clusterTest

@ajeetj ajeetj requested a review from sougou as a code owner December 5, 2019 15:53
@deepthi
Copy link
Member

deepthi commented Dec 6, 2019

rebase will fix the unit test. Not sure why cluster_endtoend is failing.

@arindamnayak arindamnayak force-pushed the tal_test_master branch 4 times, most recently from aa63289 to f37846e Compare December 6, 2019 08:21
@morgo
Copy link
Contributor

morgo commented Dec 9, 2019

@ajeetj can you move the obsolete python tests to shard 5 in test/config.json? That way we can track what has migrated. Although I would like to delete them pretty soon if the Go tests are stable.

@ajeetj
Copy link
Contributor Author

ajeetj commented Dec 9, 2019

@ajeetj can you move the obsolete python tests to shard 5 in test/config.json? That way we can track what has migrated. Although I would like to delete them pretty soon if the Go tests are stable.

@morgo Yes will do that today. I am planning a separate PR for that. Do you think we should put in this PR itself?

@morgo
Copy link
Contributor

morgo commented Dec 9, 2019

I think it is easier for reviewers to see the old files disabled in the same PR. That way it's easy to understand it's a 1:1 replacement, and not a subset.

@ajeetj
Copy link
Contributor Author

ajeetj commented Dec 9, 2019

@systay Address your comment

@morgo Updated the config.json file. Unit tests are failing after I pushed my changes. Can you re-run them? (I do not have access to rerun)

@sougou Can you please review this PR?

@sougou
Copy link
Contributor

sougou commented Dec 9, 2019

Looking good. I'll merge once you fix the DCO.

rafael and others added 11 commits December 10, 2019 13:21
@rafael @arindamnayak
Enables vreplication to run directly from MySQL
92340d4 
* Adds support for VStream to start from filename:pos and not gtid sets.
* Adds support for statement based replication streams (this should only be used
  in the context of mysql streamer, it is not safe for tablet vreplicaiton).
* Adds support to run vstream from mysql directly

Signed-off-by: Rafael Chacon <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@rafael @arindamnayak
Adds the core of vtshovel program
3b99f5b 
* Adds binary to run vtshovel.
* At the moment only working in ephemeral mode (i.e no data is persisted back to
  vrsettings).
* vtshovel only works for statement based replication right now. This is due to
  now having a good way to have a schema loader. We will itereate on this.

Signed-off-by: Rafael Chacon <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@rafael @arindamnayak
Simplifies vtshovel logic. It assumes that it runs directly again
f94dd1f 
Signed-off-by: Rafael Chacon <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@rafael @arindamnayak
Update approach to not require another binary to run vtshovel
1846888 
Signed-off-by: Rafael Chacon <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@rafael @arindamnayak
Fixes some bugs in dbconfigs and vstreamer client after inital testing
0c40216 
Signed-off-by: Rafael Chacon <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@rafael @arindamnayak
WIP: Adds test for vstreamer client
064e374 
Signed-off-by: Rafael Chacon <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@rafael @arindamnayak
Adds tests for vstreamer_client
eeb8b9c 
Signed-off-by: Rafael Chacon <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@rafael @arindamnayak
Do not pass source conn params around
e8ba26e 
* At the moment we only support erpel user. Passing source conn params around
was adding unnecessary complexity.
* This cleans up that and makes it more explicit that only erepl user is
supported. In the future we will add more flexibility in terms of what kind of
users can be configured for external vreplication streams

Signed-off-by: Rafael Chacon <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@rafael @arindamnayak
Style improvements
e92c6a4 
* Fix typo in some comments.
* Make VReplicator private again. This change is no longer needed. Originally we
wanted "vtshovel" to be an external process. Given that this now hooks into the
existent engine, there is no need to make this public.

Signed-off-by: Rafael Chacon <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@rafael @arindamnayak
Fixes per rebase with file:pos feature
b4d39e3 
Signed-off-by: Rafael Chacon <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@morgo @arindamnayak
Remove unused cruft
640b728 
Signed-off-by: Morgan Tocker <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
saurabh408 and others added 23 commits December 10, 2019 13:21
@saurabh408 @arindamnayak
Added testcase for mysqlctl process
7c5b3fc 
Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@saurabh408 @arindamnayak
removed unused variables
50608ba 
Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@saurabh408 @arindamnayak
removing comment
ead0978 
Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@saurabh408 @arindamnayak
addressed review comments
92ff978 
Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
removed cleandir flag from vttablet teardown
7d815d8 
Signed-off-by: Arindam Nayak <[email protected]>
@saurabh408 @arindamnayak
vtgate buffer testcase migrated in go
038c480 
Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@ajeetj @arindamnayak
TabletManager remaining test cases
81e03e8 
TabletManager remaining test cases in GO
Signed-off-by: Ajeet jain <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
moved sql start to non-blocking mode (#35)
5968626 
* moved sql start to non-blocking mode

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
added testcase for vtgate vars
5306db9 
Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
redirect process errors to log files (#38)
5a2a8f1 
Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
debug the test error in ci
153b6c5 
Signed-off-by: Arindam Nayak <[email protected]>
@ajeetj @arindamnayak
Tabletmanager2 test cases in GO using cluster (#23)
a8c521b 
* tabletmanager lock unlock table test case

Signed-off-by: Ajeet jain <[email protected]>

Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
Vtctld Test cases in Go using cluster (#25)
96cd10c 
* converted vtctld_test.py to go

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
Converted schema.py testcase (#26)
bbc03d2 
* migrated one of testcase from schema.py to schema_test.go

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@ajeetj @arindamnayak
Tablet Manager test cases in Go using cluster (#27)
7e93e21 
* converted tabletmanager test cases to go

Signed-off-by: Ajeet jain <[email protected]>

Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
Converted keyspace test to go (#28)
5858f65 
* ported testcase of keyspace_test.py

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
Converted sharded test from py to go (#29)
6a5b1d2 
* Converted sharded test from py to go

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@saurabh408 @arindamnayak
Added testcase for mysqlctl process
8590a22 
Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@saurabh408 @arindamnayak
removed unused variables
0def26f 
Signed-off-by: saurabh <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
removed cleandir flag from vttablet teardown
1cc8379 
Signed-off-by: Arindam Nayak <[email protected]>
@ajeetj @arindamnayak
TabletManager remaining test cases
afa32c6 
TabletManager remaining test cases in GO
Signed-off-by: Ajeet jain <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
moved sql start to non-blocking mode (#35)
f09f3be 
* moved sql start to non-blocking mode

Signed-off-by: Arindam Nayak <[email protected]>
Signed-off-by: Arindam Nayak <[email protected]>
@ajeetj @arindamnayak
Ignoring Python tests which are moved to GO (#40)
8d20666 
Signed-off-by: Ajeet jain <[email protected]>

* readme for go endtoend test cases

Signed-off-by: Ajeet jain <[email protected]>

* Update README.md

Signed-off-by: Arindam Nayak <[email protected]>
@arindamnayak
resolving merge conflict
0edbf69 
Signed-off-by: Arindam Nayak <[email protected]>
@ajeetj
Copy link
Contributor Author

ajeetj commented Dec 10, 2019

Looking good. I'll merge once you fix the DCO.

@sougou DCO is fixed.
Can you please merge this PR?

@morgo morgo merged commit 6439799 into vitessio:master Dec 10, 2019
@rafael rafael mentioned this pull request Dec 11, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@systay systay systay left review comments

@morgo morgo morgo approved these changes

@sougou sougou Awaiting requested review from sougou

@derekperkins derekperkins Awaiting requested review from derekperkins

@dkhenry dkhenry Awaiting requested review from dkhenry

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@ajeetj @deepthi @morgo @sougou @systay @rafael @arindamnayak @saurabh408