From e59dc9aacd15f76470271a95307a0ec351c8772f Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 30 Jul 2020 17:55:15 +0100 Subject: [PATCH 1/5] Fixup worker doc (again) --- docs/workers.md | 32 +++++++++++++++++--------------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/docs/workers.md b/docs/workers.md index 80b65a0cec2a..fe6d1c373e41 100644 --- a/docs/workers.md +++ b/docs/workers.md @@ -23,7 +23,7 @@ The processes communicate with each other via a Synapse-specific protocol called feeds streams of newly written data between processes so they can be kept in sync with the database state. -When configured to do so, Synapse uses a +When configured to do so, Synapse uses a [Redis pub/sub channel](https://redis.io/topics/pubsub) to send the replication stream between all configured Synapse processes. Additionally, processes may make HTTP requests to each other, primarily for operations which need to wait @@ -66,23 +66,28 @@ https://hub.docker.com/r/matrixdotorg/synapse/. To make effective use of the workers, you will need to configure an HTTP reverse-proxy such as nginx or haproxy, which will direct incoming requests to -the correct worker, or to the main synapse instance. See +the correct worker, or to the main synapse instance. See [reverse_proxy.md](reverse_proxy.md) for information on setting up a reverse proxy. -To enable workers you should create a configuration file for each worker -process. Each worker configuration file inherits the configuration of the shared -homeserver configuration file. You can then override configuration specific to -that worker, e.g. the HTTP listener that it provides (if any); logging -configuration; etc. You should minimise the number of overrides though to -maintain a usable config. +When using workers, each worker process has its own configuration file which +contains settings specific to that worker, such as the HTTP listener that it +provides (if any), logging configuration, etc. +Normally, the worker processes are configured to read from a shared +configuration file as well as the worker-specific configuration files. This +makes it easier to keep common configuration settings synchronised across all the +processes. -### Shared Configuration +The main process is somewhat special in this respect: it does not normally +need its own configuration file and can take all of its configuration from the +shared configuration file. + +Normally, only a couple of changes are needed to make an existing configuration +file suitable for use with workers. First, you need to enable an "HTTP replication +listener" for the main process; and secondly, you need to enable redis-based +replication. For example: -Next you need to add both a HTTP replication listener, used for HTTP requests -between processes, and redis config to the shared Synapse configuration file -(`homeserver.yaml`). For example: ```yaml # extend the existing `listeners` section. This defines the ports that the @@ -104,9 +109,6 @@ See the sample config for the full documentation of each option. Under **no circumstances** should the replication listener be exposed to the public internet; it has no authentication and is unencrypted. - -### Worker Configuration - In the config file for each worker, you must specify the type of worker application (`worker_app`), and you should specify a unqiue name for the worker (`worker_name`). The currently available worker applications are listed below. From f58136613766e9c31fe902f268da05d58e436f26 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Thu, 30 Jul 2020 17:56:49 +0100 Subject: [PATCH 2/5] Newsfile --- changelog.d/8000.doc | 1 + 1 file changed, 1 insertion(+) create mode 100644 changelog.d/8000.doc diff --git a/changelog.d/8000.doc b/changelog.d/8000.doc new file mode 100644 index 000000000000..8d8fd926e93c --- /dev/null +++ b/changelog.d/8000.doc @@ -0,0 +1 @@ +Improve workers docs. From 8d336921904799d2e6636f5402edec9d6892386e Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Fri, 31 Jul 2020 09:29:52 +0100 Subject: [PATCH 3/5] Add back headings --- docs/workers.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/docs/workers.md b/docs/workers.md index fe6d1c373e41..5ca5c32d4d94 100644 --- a/docs/workers.md +++ b/docs/workers.md @@ -74,15 +74,18 @@ When using workers, each worker process has its own configuration file which contains settings specific to that worker, such as the HTTP listener that it provides (if any), logging configuration, etc. -Normally, the worker processes are configured to read from a shared +Normally, the worker processes are configured to read from a shared configuration file as well as the worker-specific configuration files. This -makes it easier to keep common configuration settings synchronised across all the -processes. +makes it easier to keep common configuration settings synchronised across all +the processes. The main process is somewhat special in this respect: it does not normally need its own configuration file and can take all of its configuration from the shared configuration file. + +### Shared configuration + Normally, only a couple of changes are needed to make an existing configuration file suitable for use with workers. First, you need to enable an "HTTP replication listener" for the main process; and secondly, you need to enable redis-based @@ -109,6 +112,9 @@ See the sample config for the full documentation of each option. Under **no circumstances** should the replication listener be exposed to the public internet; it has no authentication and is unencrypted. + +### Worker configuration + In the config file for each worker, you must specify the type of worker application (`worker_app`), and you should specify a unqiue name for the worker (`worker_name`). The currently available worker applications are listed below. @@ -147,6 +153,9 @@ plain HTTP endpoint on port 8083 separately serving various endpoints, e.g. Obviously you should configure your reverse-proxy to route the relevant endpoints to the worker (`localhost:8083` in the above example). + +### Running Synapse with workers + Finally, you need to start your worker processes. This can be done with either `synctl` or your distribution's preferred service manager such as `systemd`. We recommend the use of `systemd` where available: for information on setting up From 5a249e57cc7737cfbb6fa3c1ed9c66d6246728b5 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Mon, 3 Aug 2020 13:57:26 +0100 Subject: [PATCH 4/5] Update systemd worker example --- docs/systemd-with-workers/workers/federation_reader.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/systemd-with-workers/workers/federation_reader.yaml b/docs/systemd-with-workers/workers/federation_reader.yaml index 5b65c7040d54..13e69e62c9db 100644 --- a/docs/systemd-with-workers/workers/federation_reader.yaml +++ b/docs/systemd-with-workers/workers/federation_reader.yaml @@ -1,7 +1,7 @@ worker_app: synapse.app.federation_reader +worker_name: federation_reader1 worker_replication_host: 127.0.0.1 -worker_replication_port: 9092 worker_replication_http_port: 9093 worker_listeners: From da558859efb8c255e4d9312568a327495dddbb93 Mon Sep 17 00:00:00 2001 From: Erik Johnston Date: Tue, 4 Aug 2020 09:45:25 +0100 Subject: [PATCH 5/5] Add migration notes --- docs/workers.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/workers.md b/docs/workers.md index 5ca5c32d4d94..bfec745897c2 100644 --- a/docs/workers.md +++ b/docs/workers.md @@ -418,6 +418,23 @@ all these to be folded into the `generic_worker` app and to use config to define which processes handle the various proccessing such as push notifications. +## Migration from old config + +There are two main independent changes that have been made: introducing Redis +support and merging apps into `synapse.app.generic_worker`. Both these changes +are backwards compatible and so no changes to the config are required, however +server admins are encouraged to plan to migrate to Redis as the old style direct +TCP replication config is deprecated. + +To migrate to Redis add the `redis` config as above, and optionally remove the +TCP `replication` listener from master and `worker_replication_port` from worker +config. + +To migrate apps to use `synapse.app.generic_worker` simply update the +`worker_app` option in the worker configs, and where worker are started (e.g. +in systemd service files, but not required for synctl). + + ## Architectural diagram The following shows an example setup using Redis and a reverse proxy: