Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add a Dockerfile for synapse #2846

Merged
merged 52 commits into from
May 11, 2018
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
95e02b8
docker: Initial Dockerfile and docker-compose.yaml
superdump Sep 28, 2017
24d1628
docker: s/matrix-org/matrixdotorg/g
superdump Sep 29, 2017
431476f
Initial commit including a Dockerfile for synapse
kaiyou Feb 3, 2018
d434ae3
Add template config files for the Docker image
kaiyou Feb 3, 2018
48bc22f
Allow for a wheel cache and include missing files in the build
kaiyou Feb 4, 2018
6d1e28a
Generate any missing keys before starting synapse
kaiyou Feb 4, 2018
f2bf0cd
Generate shared secrets if not defined in the environment
kaiyou Feb 4, 2018
886c2d5
Support an external postgresql config in the Docker image
kaiyou Feb 4, 2018
042757f
Install the postgres dependencies
kaiyou Feb 4, 2018
1ba2fe1
Provide an example docker compose file
kaiyou Feb 4, 2018
a207ccc
Reuse environment variables of the postgres container
kaiyou Feb 4, 2018
5396533
Merge remote-tracking branch 'origin/rob/docker' into feat-dockerfile
kaiyou Feb 4, 2018
84a9209
Remove etc/service files from rob's branch
kaiyou Feb 4, 2018
9a87b8a
Update sumperdump Docker readme to match this image properties
kaiyou Feb 4, 2018
b8ab78b
Add the build cache/ folder to gitignore
kaiyou Feb 4, 2018
f72c9c1
Fix multiple typos
kaiyou Feb 4, 2018
e9021e1
Run the server as an unprivileged user
kaiyou Feb 4, 2018
8db84e9
Remove docker related files from the python manifest
kaiyou Feb 5, 2018
81010a1
Add dynamic recaptcha configuration in the Docker image
kaiyou Feb 5, 2018
cd51931
Add dynamic TURN configuration in the Docker image
kaiyou Feb 5, 2018
cf4ef60
Document the cache factor environment variable for Docker
kaiyou Feb 5, 2018
d8c7da5
Fix a typo in the Docker README
kaiyou Feb 5, 2018
f5364b4
Point to the 'latest' tag in the Docker documentation
kaiyou Feb 5, 2018
630573a
Do not copy documentation files to the Docker root folder
kaiyou Feb 5, 2018
ee3b160
Only generate configuration files when necessary
kaiyou Feb 5, 2018
107a5c9
Add the non-tls port to the expose list
kaiyou Feb 5, 2018
1ffd9cb
Support loading application service files from /data/appservices/
kaiyou Feb 5, 2018
63fd148
Make it clear that two modes are avaiable in the documentation, impro…
kaiyou Feb 8, 2018
58df3a8
Add some documentation about high performance storage
kaiyou Feb 8, 2018
084afbb
Rename the permissions variable to avoid confusion
kaiyou Feb 8, 2018
b8a4dce
Refactor the start script to better handle mandatory parameters
kaiyou Feb 8, 2018
e174c46
Use 'synapse' as a default postgres user in Docker examples
kaiyou Feb 8, 2018
914a59c
Disable the Web client in the Docker image
kaiyou Feb 8, 2018
a0af005
Honor the SYNAPSE_REPORT_STATS parameter in the Docker image
kaiyou Feb 8, 2018
ef1f8d4
Enable email server configuration from environment variables
kaiyou Feb 8, 2018
b9b668e
Update to Alpine 3.7 and switch to libressl
kaiyou Feb 8, 2018
d8680c9
Make it clear that the image has two modes of operation
kaiyou Feb 8, 2018
48e2c64
Specify the Docker registry in the build tag
kaiyou Feb 8, 2018
a03c382
Specify the Docker registry for the postgres image
kaiyou Feb 8, 2018
e511979
Make SYNAPSE_MACAROON_SECRET_KEY a mandatory option
kaiyou Feb 8, 2018
ca70148
Fix the path to the log config file
kaiyou Feb 8, 2018
6f0b1f8
Generate macaroon and registration secrets, then store the results to…
kaiyou Feb 9, 2018
b815aa0
Remove an accidentally committed test configuration
kaiyou Feb 10, 2018
07f1b71
Explicitely provide the postgres password to synapse in the Compose e…
kaiyou Feb 10, 2018
f44b7c0
Disable logging to file and rely on the console when using Docker
kaiyou Feb 10, 2018
757f1b5
Merge remote-tracking branch 'upstream/master' into feat-dockerfile
kaiyou Mar 17, 2018
a13b786
Merge remote-tracking branch 'upstream/master' into feat-dockerfile
kaiyou Apr 8, 2018
041b41a
Merge remote-tracking branch 'upstream/master' into feat-dockerfile
kaiyou Apr 14, 2018
d4c14e1
Fix the documentation about 'POSTGRES_DB'
kaiyou May 1, 2018
4f2e898
Make the logging level configurable
kaiyou May 1, 2018
9a779c2
Merge remote-tracking branch 'upstream/master' into feat-dockerfile
kaiyou May 2, 2018
5addeaa
Add Docker packaging in the author list
kaiyou May 4, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Dockerfile
.travis.yml
.gitignore
demo/etc
tox.ini
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line seems out of place

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean tox.ini, tests are not run in the Docker image. I guess we could include the tox configuration of course, but I did not find it useful.

Copy link

@dali99 dali99 Feb 19, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright then, makes sense

1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ demo/media_store.*
demo/etc

uploads
cache

.idea/
media_store/
Expand Down
19 changes: 19 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
FROM docker.io/python:2-alpine3.7

RUN apk add --no-cache --virtual .nacl_deps su-exec build-base libffi-dev zlib-dev libressl-dev libjpeg-turbo-dev linux-headers postgresql-dev

COPY . /synapse

# A wheel cache may be provided in ./cache for faster build
RUN cd /synapse \
&& pip install --upgrade pip setuptools psycopg2 \
&& mkdir -p /synapse/cache \
&& pip install -f /synapse/cache --upgrade --process-dependency-links . \
&& mv /synapse/contrib/docker/start.py /synapse/contrib/docker/conf / \
&& rm -rf setup.py setup.cfg synapse

VOLUME ["/data"]

EXPOSE 8008/tcp 8448/tcp

ENTRYPOINT ["/start.py"]
2 changes: 2 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ recursive-include synapse/static *.js
exclude jenkins.sh
exclude jenkins*.sh
exclude jenkins*
exclude Dockerfile
exclude .dockerignore
recursive-exclude jenkins *.sh

prune .github
Expand Down
148 changes: 148 additions & 0 deletions contrib/docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# Synapse Docker

This Docker image will run Synapse as a single process. It does not provide any
database server or TURN server that you should run separately.

If you run a Postgres server, you should simply have it in the same Compose
project or set the proper environment variables and the image will automatically
use that server.

## Build

Build the docker image with the `docker build` command from the root of the synapse repository.

```
docker build -t docker.io/matrixdotorg/synapse .
```

The `-t` option sets the image tag. Official images are tagged `matrixdotorg/synapse:<version>` where `<version>` is the same as the release tag in the synapse git repository.

You may have a local Python wheel cache available, in which case copy the relevant packages in the ``cache/`` directory at the root of the project.

## Run

This image is designed to run either with an automatically generated configuration
file or with a custom configuration that requires manual edition.

### Automated configuration

It is recommended that you use Docker Compose to run your containers, including
this image and a Postgres server. A sample ``docker-compose.yml`` is provided,
including example labels for reverse proxying and other artifacts.

Read the section about environment variables and set at least mandatory variables,
then run the server:

```
docker-compose up -d
```

### Manual configuration

A sample ``docker-compose.yml`` is provided, including example labels for
reverse proxying and other artifacts.

Specify a ``SYNAPSE_CONFIG_PATH``, preferably to a persistent path,
to use manual configuration. To generate a fresh ``homeserver.yaml``, simply run:

```
docker-compose run --rm -e SYNAPSE_SERVER_NAME=my.matrix.host synapse generate
```

Then, customize your configuration and run the server:

```
docker-compose up -d
```

### Without Compose

If you do not wish to use Compose, you may still run this image using plain
Docker commands. Note that the following is just a guideline and you may need
to add parameters to the docker run command to account for the network situation
with your postgres database.

```
docker run \
-d \
--name synapse \
-v ${DATA_PATH}:/data \
-e SYNAPSE_SERVER_NAME=my.matrix.host \
-e SYNAPSE_REPORT_STATS=yes \
docker.io/matrixdotorg/synapse:latest
```

## Volumes

The image expects a single volume, located at ``/data``, that will hold:

* temporary files during uploads;
* uploaded media and thumbnails;
* the SQLite database if you do not configure postgres;
* the appservices configuration.

You are free to use separate volumes depending on storage endpoints at your
disposal. For instance, ``/data/media`` coud be stored on a large but low
performance hdd storage while other files could be stored on high performance
endpoints.

In order to setup an application service, simply create an ``appservices``
directory in the data volume and write the application service Yaml
configuration file there. Multiple application services are supported.

## Environment

Unless you specify a custom path for the configuration file, a very generic
file will be generated, based on the following environment settings.
These are a good starting point for setting up your own deployment.

Global settings:

* ``UID``, the user id Synapse will run as [default 991]
* ``GID``, the group id Synapse will run as [default 991]
* ``SYNAPSE_CONFIG_PATH``, path to a custom config file

If ``SYNAPSE_CONFIG_PATH`` is set, you should generate a configuration file
then customize it manually. No other environment variable is required.

Otherwise, a dynamic configuration file will be used. The following environment
variables are available for configuration:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it gets the point across fine, I'd put the two modes of operation somewhere more visible. People are likely to overlook it (buried in the env var list) and will likely be confused by this when things don't behave as they expect.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just pushed some documentation update about this.


* ``SYNAPSE_SERVER_NAME`` (mandatory), the current server public hostname.
* ``SYNAPSE_REPORT_STATS``, (mandatory, ``yes`` or ``no``), enable anonymous
statistics reporting back to the Matrix project which helps us to get funding.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it "no"?

* ``SYNAPSE_MACAROON_SECRET_KEY`` (mandatory) secret for signing access tokens
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iirc this can't change after initially being set otherwise synapse can't understand previous access tokens.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I did hesitate with a default behavior generating a random value and storing it to the data dir.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some added documentation to say "don't ever change this" would be nice :)

to the server, set this to a proper random key.
* ``SYNAPSE_NO_TLS``, set this variable to disable TLS in Synapse (use this if
you run your own TLS-capable reverse proxy).
* ``SYNAPSE_ENABLE_REGISTRATION``, set this variable to enable registration on
the Synapse instance.
* ``SYNAPSE_ALLOW_GUEST``, set this variable to allow guest joining this server.
* ``SYNAPSE_EVENT_CACHE_SIZE``, the event cache size [default `10K`].
* ``SYNAPSE_CACHE_FACTOR``, the cache factor [default `0.5`].
* ``SYNAPSE_RECAPTCHA_PUBLIC_KEY``, set this variable to the recaptcha public
key in order to enable recaptcha upon registration.
* ``SYNAPSE_RECAPTCHA_PRIVATE_KEY``, set this variable to the recaptcha private
key in order to enable recaptcha upon registration.
* ``SYNAPSE_TURN_URIS``, set this variable to the coma-separated list of TURN
uris to enable TURN for this homeserver.
* ``SYNAPSE_TURN_SECRET``, set this to the TURN shared secret if required.

Shared secrets, that will be initialized to random values if not set:

* ``SYNAPSE_REGISTRATION_SHARED_SECRET``, secret for registrering users if
registration is disable.

Database specific values (will use SQLite if not set):

* `POSTGRES_DATABASE` - The database name for the synapse postgres database. [default: `matrix`]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If so, the default also seems to be "synapse" too.

* `POSTGRES_HOST` - The host of the postgres database if you wish to use postgresql instead of sqlite3. [default: `db` which is useful when using a container on the same docker network in a compose file where the postgres service is called `db`]
* `POSTGRES_PASSWORD` - The password for the synapse postgres database. **If this is set then postgres will be used instead of sqlite3.** [default: none] **NOTE**: You are highly encouraged to use postgresql! Please use the compose file to make it easier to deploy.
* `POSTGRES_USER` - The user for the synapse postgres database. [default: `matrix`]

Mail server specific values (will not send emails if not set):

* ``SYNAPSE_SMTP_HOST``, hostname to the mail server.
* ``SYNAPSE_SMTP_PORT``, TCP port for accessing the mail server [default ``25``].
* ``SYNAPSE_SMTP_USER``, username for authenticating against the mail server if any.
* ``SYNAPSE_SMTP_PASSWORD``, password for authenticating against the mail server if any.
Loading