Skip to content
This repository has been archived by the owner on Apr 12, 2024. It is now read-only.

Commit

Permalink
Merge tag 'v1.49.0' into dinsic
Browse files Browse the repository at this point in the history
Synapse 1.49.0 (2021-12-14)
===========================

No significant changes since version 1.49.0rc1.

Support for Ubuntu 21.04 ends next month on the 20th of January
---------------------------------------------------------------

For users of Ubuntu 21.04 (Hirsute Hippo), please be aware that [upstream support for this version of Ubuntu will end next month][Ubuntu2104EOL].
We will stop producing packages for Ubuntu 21.04 after upstream support ends.

[Ubuntu2104EOL]: https://lists.ubuntu.com/archives/ubuntu-announce/2021-December/000275.html

The wiki has been migrated to the documentation website
-------------------------------------------------------

We've decided to move the existing, somewhat stagnant pages from the GitHub wiki
to the [documentation website](https://matrix-org.github.io/synapse/latest/).

This was done for two reasons. The first was to ensure that changes are checked by
multiple authors before being committed (everyone makes mistakes!) and the second
was visibility of the documentation. Not everyone knows that Synapse has some very
useful information hidden away in its GitHub wiki pages. Bringing them to the
documentation website should help with visibility, as well as keep all Synapse documentation
in one, easily-searchable location.

Note that contributions to the documentation website happen through [GitHub pull
requests](https://github.com/matrix-org/synapse/pulls). Please visit [#synapse-dev:matrix.org](https://matrix.to/#/#synapse-dev:matrix.org)
if you need help with the process!

Synapse 1.49.0rc1 (2021-12-07)
==============================

Features
--------

- Add [MSC3030](matrix-org/matrix-spec-proposals#3030) experimental client and federation API endpoints to get the closest event to a given timestamp. ([\#9445](matrix-org/synapse#9445))
- Include bundled relation aggregations during a limited `/sync` request and `/relations` request, per [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\#11284](matrix-org/synapse#11284), [\#11478](matrix-org/synapse#11478))
- Add plugin support for controlling database background updates. ([\#11306](matrix-org/synapse#11306), [\#11475](matrix-org/synapse#11475), [\#11479](matrix-org/synapse#11479))
- Support the stable API endpoints for [MSC2946](matrix-org/matrix-spec-proposals#2946): the room `/hierarchy` endpoint. ([\#11329](matrix-org/synapse#11329))
- Add admin API to get some information about federation status with remote servers. ([\#11407](matrix-org/synapse#11407))
- Support expiry of refresh tokens and expiry of the overall session when refresh tokens are in use. ([\#11425](matrix-org/synapse#11425))
- Stabilise support for [MSC2918](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens) refresh tokens as they have now been merged into the Matrix specification. ([\#11435](matrix-org/synapse#11435), [\#11522](matrix-org/synapse#11522))
- Update [MSC2918 refresh token](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens) support to confirm with the latest revision: accept the `refresh_tokens` parameter in the request body rather than in the URL parameters. ([\#11430](matrix-org/synapse#11430))
- Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. ([\#11445](matrix-org/synapse#11445))
- Expose `synapse_homeserver` and `synapse_worker` commands as entry points to run Synapse's main process and worker processes, respectively. Contributed by @Ma27. ([\#11449](matrix-org/synapse#11449))
- `synctl stop` will now wait for Synapse to exit before returning. ([\#11459](matrix-org/synapse#11459), [\#11490](matrix-org/synapse#11490))
- Extend the "delete room" admin api to work correctly on rooms which have previously been partially deleted. ([\#11523](matrix-org/synapse#11523))
- Add support for the `/_matrix/client/v3/login/sso/redirect/{idpId}` API from Matrix v1.1. This endpoint was overlooked when support for v3 endpoints was added in Synapse 1.48.0rc1. ([\#11451](matrix-org/synapse#11451))

Bugfixes
--------

- Fix using [MSC2716](matrix-org/matrix-spec-proposals#2716) batch sending in combination with event persistence workers. Contributed by @tulir at Beeper. ([\#11220](matrix-org/synapse#11220))
- Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection, properly this time. Also fix a race condition introduced in the previous insufficient fix in Synapse 1.47.0. ([\#11376](matrix-org/synapse#11376))
- The `/send_join` response now includes the stable `event` field instead of the unstable field from [MSC3083](matrix-org/matrix-spec-proposals#3083). ([\#11413](matrix-org/synapse#11413))
- Fix a bug introduced in Synapse 1.47.0 where `send_join` could fail due to an outdated `ijson` version. ([\#11439](matrix-org/synapse#11439), [\#11441](matrix-org/synapse#11441), [\#11460](matrix-org/synapse#11460))
- Fix a bug introduced in Synapse 1.36.0 which could cause problems fetching event-signing keys from trusted key servers. ([\#11440](matrix-org/synapse#11440))
- Fix a bug introduced in Synapse 1.47.1 where the media repository would fail to work if the media store path contained any symbolic links. ([\#11446](matrix-org/synapse#11446))
- Fix an `LruCache` corruption bug, introduced in Synapse 1.38.0, that would cause certain requests to fail until the next Synapse restart. ([\#11454](matrix-org/synapse#11454))
- Fix a long-standing bug where invites from ignored users were included in incremental syncs. ([\#11511](matrix-org/synapse#11511))
- Fix a regression in Synapse 1.48.0 where presence workers would not clear their presence updates over replication on shutdown. ([\#11518](matrix-org/synapse#11518))
- Fix a regression in Synapse 1.48.0 where the module API's `looping_background_call` method would spam errors to the logs when given a non-async function. ([\#11524](matrix-org/synapse#11524))

Updates to the Docker image
---------------------------

- Update `Dockerfile-workers` to healthcheck all workers in the container. ([\#11429](matrix-org/synapse#11429))

Improved Documentation
----------------------

- Update the media repository documentation. ([\#11415](matrix-org/synapse#11415))
- Update section about backward extremities in the room DAG concepts doc to correct the misconception about backward extremities indicating whether we have fetched an events' `prev_events`. ([\#11469](matrix-org/synapse#11469))

Internal Changes
----------------

- Add `Final` annotation to string constants in `synapse.api.constants` so that they get typed as `Literal`s. ([\#11356](matrix-org/synapse#11356))
- Add a check to ensure that users cannot start the Synapse master process when `worker_app` is set. ([\#11416](matrix-org/synapse#11416))
- Add a note about postgres memory management and hugepages to postgres doc. ([\#11467](matrix-org/synapse#11467))
- Add missing type hints to `synapse.config` module. ([\#11465](matrix-org/synapse#11465))
- Add missing type hints to `synapse.federation`. ([\#11483](matrix-org/synapse#11483))
- Add type annotations to `tests.storage.test_appservice`. ([\#11488](matrix-org/synapse#11488), [\#11492](matrix-org/synapse#11492))
- Add type annotations to some of the configuration surrounding refresh tokens. ([\#11428](matrix-org/synapse#11428))
- Add type hints to `synapse/tests/rest/admin`. ([\#11501](matrix-org/synapse#11501))
- Add type hints to storage classes. ([\#11411](matrix-org/synapse#11411))
- Add wiki pages to documentation website. ([\#11402](matrix-org/synapse#11402))
- Clean up `tests.storage.test_main` to remove use of legacy code. ([\#11493](matrix-org/synapse#11493))
- Clean up `tests.test_visibility` to remove legacy code. ([\#11495](matrix-org/synapse#11495))
- Convert status codes to `HTTPStatus` in `synapse.rest.admin`. ([\#11452](matrix-org/synapse#11452), [\#11455](matrix-org/synapse#11455))
- Extend the `scripts-dev/sign_json` script to support signing events. ([\#11486](matrix-org/synapse#11486))
- Improve internal types in push code. ([\#11409](matrix-org/synapse#11409))
- Improve type annotations in `synapse.module_api`. ([\#11029](matrix-org/synapse#11029))
- Improve type hints for `LruCache`. ([\#11453](matrix-org/synapse#11453))
- Preparation for database schema simplifications: disambiguate queries on `state_key`. ([\#11497](matrix-org/synapse#11497))
- Refactor `backfilled` into specific behavior function arguments (`_persist_events_and_state_updates` and downstream calls). ([\#11417](matrix-org/synapse#11417))
- Refactor `get_version_string` to fix-up types and duplicated code. ([\#11468](matrix-org/synapse#11468))
- Refactor various parts of the `/sync` handler. ([\#11494](matrix-org/synapse#11494), [\#11515](matrix-org/synapse#11515))
- Remove unnecessary `json.dumps` from `tests.rest.admin`. ([\#11461](matrix-org/synapse#11461))
- Save the OpenID Connect session ID on login. ([\#11482](matrix-org/synapse#11482))
- Update and clean up recently ported documentation pages. ([\#11466](matrix-org/synapse#11466))
  • Loading branch information
babolivier committed Feb 7, 2022
2 parents 783f031 + 92906e1 commit 3ef7c0c
Show file tree
Hide file tree
Showing 165 changed files with 7,745 additions and 2,709 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,7 @@ jobs:
working-directory: complement/dockerfiles

# Run Complement
- run: go test -v -tags synapse_blacklist,msc2403,msc2946,msc3083 ./tests/...
- run: go test -v -tags synapse_blacklist,msc2403 ./tests/...
env:
COMPLEMENT_BASE_IMAGE: complement-synapse:latest
working-directory: complement
Expand Down
111 changes: 111 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,114 @@
Synapse 1.49.0 (2021-12-14)
===========================

No significant changes since version 1.49.0rc1.


Support for Ubuntu 21.04 ends next month on the 20th of January
---------------------------------------------------------------

For users of Ubuntu 21.04 (Hirsute Hippo), please be aware that [upstream support for this version of Ubuntu will end next month][Ubuntu2104EOL].
We will stop producing packages for Ubuntu 21.04 after upstream support ends.

[Ubuntu2104EOL]: https://lists.ubuntu.com/archives/ubuntu-announce/2021-December/000275.html


The wiki has been migrated to the documentation website
-------------------------------------------------------

We've decided to move the existing, somewhat stagnant pages from the GitHub wiki
to the [documentation website](https://matrix-org.github.io/synapse/latest/).

This was done for two reasons. The first was to ensure that changes are checked by
multiple authors before being committed (everyone makes mistakes!) and the second
was visibility of the documentation. Not everyone knows that Synapse has some very
useful information hidden away in its GitHub wiki pages. Bringing them to the
documentation website should help with visibility, as well as keep all Synapse documentation
in one, easily-searchable location.

Note that contributions to the documentation website happen through [GitHub pull
requests](https://github.com/matrix-org/synapse/pulls). Please visit [#synapse-dev:matrix.org](https://matrix.to/#/#synapse-dev:matrix.org)
if you need help with the process!


Synapse 1.49.0rc1 (2021-12-07)
==============================

Features
--------

- Add [MSC3030](https://github.com/matrix-org/matrix-doc/pull/3030) experimental client and federation API endpoints to get the closest event to a given timestamp. ([\#9445](https://github.com/matrix-org/synapse/issues/9445))
- Include bundled relation aggregations during a limited `/sync` request and `/relations` request, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675). ([\#11284](https://github.com/matrix-org/synapse/issues/11284), [\#11478](https://github.com/matrix-org/synapse/issues/11478))
- Add plugin support for controlling database background updates. ([\#11306](https://github.com/matrix-org/synapse/issues/11306), [\#11475](https://github.com/matrix-org/synapse/issues/11475), [\#11479](https://github.com/matrix-org/synapse/issues/11479))
- Support the stable API endpoints for [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946): the room `/hierarchy` endpoint. ([\#11329](https://github.com/matrix-org/synapse/issues/11329))
- Add admin API to get some information about federation status with remote servers. ([\#11407](https://github.com/matrix-org/synapse/issues/11407))
- Support expiry of refresh tokens and expiry of the overall session when refresh tokens are in use. ([\#11425](https://github.com/matrix-org/synapse/issues/11425))
- Stabilise support for [MSC2918](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens) refresh tokens as they have now been merged into the Matrix specification. ([\#11435](https://github.com/matrix-org/synapse/issues/11435), [\#11522](https://github.com/matrix-org/synapse/issues/11522))
- Update [MSC2918 refresh token](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens) support to confirm with the latest revision: accept the `refresh_tokens` parameter in the request body rather than in the URL parameters. ([\#11430](https://github.com/matrix-org/synapse/issues/11430))
- Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. ([\#11445](https://github.com/matrix-org/synapse/issues/11445))
- Expose `synapse_homeserver` and `synapse_worker` commands as entry points to run Synapse's main process and worker processes, respectively. Contributed by @Ma27. ([\#11449](https://github.com/matrix-org/synapse/issues/11449))
- `synctl stop` will now wait for Synapse to exit before returning. ([\#11459](https://github.com/matrix-org/synapse/issues/11459), [\#11490](https://github.com/matrix-org/synapse/issues/11490))
- Extend the "delete room" admin api to work correctly on rooms which have previously been partially deleted. ([\#11523](https://github.com/matrix-org/synapse/issues/11523))
- Add support for the `/_matrix/client/v3/login/sso/redirect/{idpId}` API from Matrix v1.1. This endpoint was overlooked when support for v3 endpoints was added in Synapse 1.48.0rc1. ([\#11451](https://github.com/matrix-org/synapse/issues/11451))


Bugfixes
--------

- Fix using [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) batch sending in combination with event persistence workers. Contributed by @tulir at Beeper. ([\#11220](https://github.com/matrix-org/synapse/issues/11220))
- Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection, properly this time. Also fix a race condition introduced in the previous insufficient fix in Synapse 1.47.0. ([\#11376](https://github.com/matrix-org/synapse/issues/11376))
- The `/send_join` response now includes the stable `event` field instead of the unstable field from [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083). ([\#11413](https://github.com/matrix-org/synapse/issues/11413))
- Fix a bug introduced in Synapse 1.47.0 where `send_join` could fail due to an outdated `ijson` version. ([\#11439](https://github.com/matrix-org/synapse/issues/11439), [\#11441](https://github.com/matrix-org/synapse/issues/11441), [\#11460](https://github.com/matrix-org/synapse/issues/11460))
- Fix a bug introduced in Synapse 1.36.0 which could cause problems fetching event-signing keys from trusted key servers. ([\#11440](https://github.com/matrix-org/synapse/issues/11440))
- Fix a bug introduced in Synapse 1.47.1 where the media repository would fail to work if the media store path contained any symbolic links. ([\#11446](https://github.com/matrix-org/synapse/issues/11446))
- Fix an `LruCache` corruption bug, introduced in Synapse 1.38.0, that would cause certain requests to fail until the next Synapse restart. ([\#11454](https://github.com/matrix-org/synapse/issues/11454))
- Fix a long-standing bug where invites from ignored users were included in incremental syncs. ([\#11511](https://github.com/matrix-org/synapse/issues/11511))
- Fix a regression in Synapse 1.48.0 where presence workers would not clear their presence updates over replication on shutdown. ([\#11518](https://github.com/matrix-org/synapse/issues/11518))
- Fix a regression in Synapse 1.48.0 where the module API's `looping_background_call` method would spam errors to the logs when given a non-async function. ([\#11524](https://github.com/matrix-org/synapse/issues/11524))


Updates to the Docker image
---------------------------

- Update `Dockerfile-workers` to healthcheck all workers in the container. ([\#11429](https://github.com/matrix-org/synapse/issues/11429))


Improved Documentation
----------------------

- Update the media repository documentation. ([\#11415](https://github.com/matrix-org/synapse/issues/11415))
- Update section about backward extremities in the room DAG concepts doc to correct the misconception about backward extremities indicating whether we have fetched an events' `prev_events`. ([\#11469](https://github.com/matrix-org/synapse/issues/11469))


Internal Changes
----------------

- Add `Final` annotation to string constants in `synapse.api.constants` so that they get typed as `Literal`s. ([\#11356](https://github.com/matrix-org/synapse/issues/11356))
- Add a check to ensure that users cannot start the Synapse master process when `worker_app` is set. ([\#11416](https://github.com/matrix-org/synapse/issues/11416))
- Add a note about postgres memory management and hugepages to postgres doc. ([\#11467](https://github.com/matrix-org/synapse/issues/11467))
- Add missing type hints to `synapse.config` module. ([\#11465](https://github.com/matrix-org/synapse/issues/11465))
- Add missing type hints to `synapse.federation`. ([\#11483](https://github.com/matrix-org/synapse/issues/11483))
- Add type annotations to `tests.storage.test_appservice`. ([\#11488](https://github.com/matrix-org/synapse/issues/11488), [\#11492](https://github.com/matrix-org/synapse/issues/11492))
- Add type annotations to some of the configuration surrounding refresh tokens. ([\#11428](https://github.com/matrix-org/synapse/issues/11428))
- Add type hints to `synapse/tests/rest/admin`. ([\#11501](https://github.com/matrix-org/synapse/issues/11501))
- Add type hints to storage classes. ([\#11411](https://github.com/matrix-org/synapse/issues/11411))
- Add wiki pages to documentation website. ([\#11402](https://github.com/matrix-org/synapse/issues/11402))
- Clean up `tests.storage.test_main` to remove use of legacy code. ([\#11493](https://github.com/matrix-org/synapse/issues/11493))
- Clean up `tests.test_visibility` to remove legacy code. ([\#11495](https://github.com/matrix-org/synapse/issues/11495))
- Convert status codes to `HTTPStatus` in `synapse.rest.admin`. ([\#11452](https://github.com/matrix-org/synapse/issues/11452), [\#11455](https://github.com/matrix-org/synapse/issues/11455))
- Extend the `scripts-dev/sign_json` script to support signing events. ([\#11486](https://github.com/matrix-org/synapse/issues/11486))
- Improve internal types in push code. ([\#11409](https://github.com/matrix-org/synapse/issues/11409))
- Improve type annotations in `synapse.module_api`. ([\#11029](https://github.com/matrix-org/synapse/issues/11029))
- Improve type hints for `LruCache`. ([\#11453](https://github.com/matrix-org/synapse/issues/11453))
- Preparation for database schema simplifications: disambiguate queries on `state_key`. ([\#11497](https://github.com/matrix-org/synapse/issues/11497))
- Refactor `backfilled` into specific behavior function arguments (`_persist_events_and_state_updates` and downstream calls). ([\#11417](https://github.com/matrix-org/synapse/issues/11417))
- Refactor `get_version_string` to fix-up types and duplicated code. ([\#11468](https://github.com/matrix-org/synapse/issues/11468))
- Refactor various parts of the `/sync` handler. ([\#11494](https://github.com/matrix-org/synapse/issues/11494), [\#11515](https://github.com/matrix-org/synapse/issues/11515))
- Remove unnecessary `json.dumps` from `tests.rest.admin`. ([\#11461](https://github.com/matrix-org/synapse/issues/11461))
- Save the OpenID Connect session ID on login. ([\#11482](https://github.com/matrix-org/synapse/issues/11482))
- Update and clean up recently ported documentation pages. ([\#11466](https://github.com/matrix-org/synapse/issues/11466))


Synapse 1.48.0 (2021-11-30)
===========================

Expand Down
12 changes: 12 additions & 0 deletions debian/changelog
Original file line number Diff line number Diff line change
@@ -1,3 +1,15 @@
matrix-synapse-py3 (1.49.0) stable; urgency=medium

* New synapse release 1.49.0.

-- Synapse Packaging team <[email protected]> Tue, 14 Dec 2021 12:39:46 +0000

matrix-synapse-py3 (1.49.0~rc1) stable; urgency=medium

* New synapse release 1.49.0~rc1.

-- Synapse Packaging team <[email protected]> Tue, 07 Dec 2021 13:52:21 +0000

matrix-synapse-py3 (1.48.0) stable; urgency=medium

* New synapse release 1.48.0.
Expand Down
3 changes: 3 additions & 0 deletions docker/Dockerfile-workers
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,6 @@ VOLUME ["/data"]
# files to run the desired worker configuration. Will start supervisord.
COPY ./docker/configure_workers_and_start.py /configure_workers_and_start.py
ENTRYPOINT ["/configure_workers_and_start.py"]

HEALTHCHECK --start-period=5s --interval=15s --timeout=5s \
CMD /bin/sh /healthcheck.sh
6 changes: 6 additions & 0 deletions docker/conf-workers/healthcheck.sh.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/sh
# This healthcheck script is designed to return OK when every
# host involved returns OK
{%- for healthcheck_url in healthcheck_urls %}
curl -fSs {{ healthcheck_url }} || exit 1
{%- endfor %}
13 changes: 13 additions & 0 deletions docker/configure_workers_and_start.py
Original file line number Diff line number Diff line change
Expand Up @@ -474,10 +474,16 @@ def generate_worker_files(environ, config_path: str, data_dir: str):

# Determine the load-balancing upstreams to configure
nginx_upstream_config = ""

# At the same time, prepare a list of internal endpoints to healthcheck
# starting with the main process which exists even if no workers do.
healthcheck_urls = ["http://localhost:8080/health"]

for upstream_worker_type, upstream_worker_ports in nginx_upstreams.items():
body = ""
for port in upstream_worker_ports:
body += " server localhost:%d;\n" % (port,)
healthcheck_urls.append("http://localhost:%d/health" % (port,))

# Add to the list of configured upstreams
nginx_upstream_config += NGINX_UPSTREAM_CONFIG_BLOCK.format(
Expand Down Expand Up @@ -510,6 +516,13 @@ def generate_worker_files(environ, config_path: str, data_dir: str):
worker_config=supervisord_config,
)

# healthcheck config
convert(
"/conf/healthcheck.sh.j2",
"/healthcheck.sh",
healthcheck_urls=healthcheck_urls,
)

# Ensure the logging directory exists
log_dir = data_dir + "/logs"
if not os.path.exists(log_dir):
Expand Down
8 changes: 8 additions & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
- [Presence router callbacks](modules/presence_router_callbacks.md)
- [Account validity callbacks](modules/account_validity_callbacks.md)
- [Password auth provider callbacks](modules/password_auth_provider_callbacks.md)
- [Background update controller callbacks](modules/background_update_controller_callbacks.md)
- [Porting a legacy module to the new interface](modules/porting_legacy_module.md)
- [Workers](workers.md)
- [Using `synctl` with Workers](synctl_workers.md)
Expand All @@ -64,9 +65,15 @@
- [Statistics](admin_api/statistics.md)
- [Users](admin_api/user_admin_api.md)
- [Server Version](admin_api/version_api.md)
- [Federation](usage/administration/admin_api/federation.md)
- [Manhole](manhole.md)
- [Monitoring](metrics-howto.md)
- [Understanding Synapse Through Grafana Graphs](usage/administration/understanding_synapse_through_grafana_graphs.md)
- [Useful SQL for Admins](usage/administration/useful_sql_for_admins.md)
- [Database Maintenance Tools](usage/administration/database_maintenance_tools.md)
- [State Groups](usage/administration/state_groups.md)
- [Request log format](usage/administration/request_log.md)
- [Admin FAQ](usage/administration/admin_faq.md)
- [Scripts]()

# Development
Expand Down Expand Up @@ -94,3 +101,4 @@

# Other
- [Dependency Deprecation Policy](deprecation_policy.md)
- [Running Synapse on a Single-Board Computer](other/running_synapse_on_single_board_computers.md)
16 changes: 7 additions & 9 deletions docs/development/room-dag-concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,15 @@ Most-recent-in-time events in the DAG which are not referenced by any other even
The forward extremities of a room are used as the `prev_events` when the next event is sent.


## Backwards extremity
## Backward extremity

The current marker of where we have backfilled up to and will generally be the
oldest-in-time events we know of in the DAG.
`prev_events` of the oldest-in-time events we have in the DAG. This gives a starting point when
backfilling history.

This is an event where we haven't fetched all of the `prev_events` for.

Once we have fetched all of its `prev_events`, it's unmarked as a backwards
extremity (although we may have formed new backwards extremities from the prev
events during the backfilling process).
When we persist a non-outlier event, we clear it as a backward extremity and set
all of its `prev_events` as the new backward extremities if they aren't already
persisted in the `events` table.


## Outliers
Expand All @@ -56,8 +55,7 @@ We mark an event as an `outlier` when we haven't figured out the state for the
room at that point in the DAG yet.

We won't *necessarily* have the `prev_events` of an `outlier` in the database,
but it's entirely possible that we *might*. The status of whether we have all of
the `prev_events` is marked as a [backwards extremity](#backwards-extremity).
but it's entirely possible that we *might*.

For example, when we fetch the event auth chain or state for a given event, we
mark all of those claimed auth events as outliers because we haven't done the
Expand Down
Loading

0 comments on commit 3ef7c0c

Please sign in to comment.