From 9904b73a47c94498b187d5851a3d11ca2601efe5 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Mon, 26 Jul 2021 17:23:24 -0500 Subject: [PATCH 1/5] Document Complement dev usage Also includes info on how to acces the databae for the homeserver after running Complement, see https://github.com/matrix-org/complement/pull/165 --- CONTRIBUTING.md | 43 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 41 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 80ef6aa235ae..d79879ea4c0d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -155,7 +155,7 @@ source ./env/bin/activate ./scripts-dev/lint.sh path/to/file1.py path/to/file2.py path/to/folder ``` -## Run the unit tests. +## Run the unit tests (Twisted trial). The unit tests run parts of Synapse, including your changes, to see if anything was broken. They are slower than the linters but will typically catch more errors. @@ -186,7 +186,7 @@ SYNAPSE_TEST_LOG_LEVEL=DEBUG trial tests ``` -## Run the integration tests. +## Run the integration tests ([Sytest](https://github.com/matrix-org/sytest)). The integration tests are a more comprehensive suite of tests. They run a full version of Synapse, including your changes, to check if @@ -203,6 +203,45 @@ $ docker run --rm -it -v /path/where/you/have/cloned/the/repository\:/src:ro -v This configuration should generally cover your needs. For more details about other configurations, see [documentation in the SyTest repo](https://github.com/matrix-org/sytest/blob/develop/docker/README.md). +## Run the integration tests ([Complement](https://github.com/matrix-org/complement)). + +[Complement](https://github.com/matrix-org/complement) is a suite of black box tests that can be run on any homeserver implementation. It can also be thought of as end-to-end (e2e) tests. + +It's often nice to develop on Synapse and write Complement tests at the same time. +Here is how to run your local Synapse checkout against your local Complement checkout. + +(checkout [`complement`](https://github.com/matrix-org/complement) alongside your `synapse` checkout) +```sh +COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh +``` + +To run a specific test file, you can adjust [`scripts-dev/complement.sh`](https://github.com/matrix-org/synapse/blob/develop/scripts-dev/complement.sh) to use the `-run MyTest` syntax. The name comes from the name structure in your Complement tests. If you're unsure of the name, you can do a full run and copy it from the test output: + +```sh +go test -v -tags synapse_blacklist,msc2946,msc2716,msc3083,msc2403 -count=1 $EXTRA_COMPLEMENT_ARGS ./tests -run TestBackfillingHistory +``` + +To run a specific test, you can specify the whole name structure: + +```sh +go test -v -tags synapse_blacklist,msc2946,msc2716,msc3083,msc2403 -count=1 $EXTRA_COMPLEMENT_ARGS ./tests -run TestBackfillingHistory/parallel/Backfilled_historical_events_resolve_with_proper_state_in_correct_order +``` + + +### Access database for homeserver after Complement test runs. + +If you're curious what the database looks like after you run some tests, here are some steps to get you going in Synapse: + + 1. In your Complement test comment out `defer deployment.Destroy(t)` and replace with `defer time.Sleep(2 * time.Hour)` to keep the homeserver running after the tests complete + 1. Start the Complement tests + 1. Find the name of the container, `docker ps -f name=complement_` (this will filter for just the Compelement related Docker containers) + 1. Access the container replacing the name with what you found in the previous step: `docker exec -it complement_1_hs_with_application_service.hs1_2 /bin/bash` + 1. Install sqlite (database driver), `apt-get update && apt-get install -y sqlite3` + 1. Then run `sqlite3` and open the database `.open /conf/homeserver.db` (this db path comes from the Synapse homeserver.yaml) + + + + # 9. Submit your patch. Once you're happy with your patch, it's time to prepare a Pull Request. From 9d82196498faf248fa40663f20a16ead5235aa00 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Mon, 26 Jul 2021 17:29:14 -0500 Subject: [PATCH 2/5] Add changelog --- changelog.d/10483.doc | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelog.d/10483.doc diff --git a/changelog.d/10483.doc b/changelog.d/10483.doc new file mode 100644 index 000000000000..0f699fafddba --- /dev/null +++ b/changelog.d/10483.doc @@ -0,0 +1 @@ +Document how to use Complement while developing a new Synapse feature. From 1ba6150f2d09f761b44d9ad265a5b37e41db9157 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Mon, 26 Jul 2021 23:16:58 -0500 Subject: [PATCH 3/5] Use extra arg syntax --- CONTRIBUTING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d79879ea4c0d..848a822c9905 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -218,13 +218,13 @@ COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh To run a specific test file, you can adjust [`scripts-dev/complement.sh`](https://github.com/matrix-org/synapse/blob/develop/scripts-dev/complement.sh) to use the `-run MyTest` syntax. The name comes from the name structure in your Complement tests. If you're unsure of the name, you can do a full run and copy it from the test output: ```sh -go test -v -tags synapse_blacklist,msc2946,msc2716,msc3083,msc2403 -count=1 $EXTRA_COMPLEMENT_ARGS ./tests -run TestBackfillingHistory +COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory ``` To run a specific test, you can specify the whole name structure: ```sh -go test -v -tags synapse_blacklist,msc2946,msc2716,msc3083,msc2403 -count=1 $EXTRA_COMPLEMENT_ARGS ./tests -run TestBackfillingHistory/parallel/Backfilled_historical_events_resolve_with_proper_state_in_correct_order +COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory/parallel/Backfilled_historical_events_resolve_with_proper_state_in_correct_order ``` From 18e03e462a4a3d577376b8c35e378c742077eb5b Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Tue, 27 Jul 2021 14:06:30 -0500 Subject: [PATCH 4/5] Update text to match being able to pass in test name --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 848a822c9905..9e6a84f3fce8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -215,7 +215,7 @@ Here is how to run your local Synapse checkout against your local Complement che COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh ``` -To run a specific test file, you can adjust [`scripts-dev/complement.sh`](https://github.com/matrix-org/synapse/blob/develop/scripts-dev/complement.sh) to use the `-run MyTest` syntax. The name comes from the name structure in your Complement tests. If you're unsure of the name, you can do a full run and copy it from the test output: +To run a specific test file, you can pass the test name at the end of the command. The name passed comes from the naming structure in your Complement tests. If you're unsure of the name, you can do a full run and copy it from the test output: ```sh COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestBackfillingHistory From 4d47e6d92bb31fba3224d2949873308cca3dbb17 Mon Sep 17 00:00:00 2001 From: Eric Eastwood Date: Tue, 27 Jul 2021 14:07:04 -0500 Subject: [PATCH 5/5] Reduce spacing --- CONTRIBUTING.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9e6a84f3fce8..e7eef23419d5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -240,8 +240,6 @@ If you're curious what the database looks like after you run some tests, here ar 1. Then run `sqlite3` and open the database `.open /conf/homeserver.db` (this db path comes from the Synapse homeserver.yaml) - - # 9. Submit your patch. Once you're happy with your patch, it's time to prepare a Pull Request.