Implementation of MSC2314 #6176

hawkowl · 2019-10-07T15:24:39Z

MSC2314 is still in proposal, but here's an implementation that I can maybe use to bootstrap rescuing my server :D

MSC: matrix-org/matrix-spec-proposals#2314

erikjohnston

Seems about right, given the MSC

…hawkowl/msc2314

richvdh

looks generally sane to me, though as per matrix-org/matrix-spec-proposals#2314 (comment) I'm not sure if it needs to be on a separate endpoint.

richvdh

lgtm modulo a couple of nits below which you might like to consider before merging

richvdh · 2019-11-27T07:22:15Z

sytest-blacklist

+
+# Blacklisted due to https://github.com/matrix-org/matrix-doc/pull/2314 removing
+# this requirement from the spec
+Inbound federation of state requires event_id as a mandatory paramater


probably a better fix is to remove the test from sytest

I guess this is debatable until the MSC lands. 🤷‍♂️

yeah the spec requires event_id as mandatory, after the MSC lands we should def remove from sytest

richvdh · 2019-11-27T07:24:24Z

tests/unittest.py

@@ -559,6 +565,59 @@ def attempt_wrong_password_login(self, username, password):
        self.render(request)
        self.assertEqual(channel.code, 403, channel.result)

+    def inject_room_member(self, room, user, membership, replaces_state=None):


docstring on this wouldn't go amiss

Synapse 1.7.0rc1 (2019-12-09) ============================= Features -------- - Implement per-room message retention policies. ([\#5815](#5815), [\#6436](#6436)) - Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](#5858)) - Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](#5925)) - Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](#6119)) - Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](#6176)) - Configure privacy-preserving settings by default for the room directory. ([\#6354](#6354)) - Add ephemeral messages support by partially implementing [MSC2228](matrix-org/matrix-spec-proposals#2228). ([\#6409](#6409)) - Add support for [MSC 2367](matrix-org/matrix-spec-proposals#2367), which allows specifying a reason on all membership events. ([\#6434](#6434)) Bugfixes -------- - Transfer non-standard power levels on room upgrade. ([\#6237](#6237)) - Fix error from the Pillow library when uploading RGBA images. ([\#6241](#6241)) - Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](#6329)) - Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](#6332)) - Prevent account data syncs getting lost across TCP replication. ([\#6333](#6333)) - Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](#6406)) - Fix an intermittent exception when handling read-receipts. ([\#6408](#6408)) - Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](#6420)) - Fix startup error when http proxy is defined. ([\#6421](#6421)) - Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](#6449)) - Fix uploading multiple cross signing signatures for the same user. ([\#6451](#6451)) - Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](#6462)) - Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](#6470)) - Improve sanity-checking when receiving events over federation. ([\#6472](#6472)) - Fix inaccurate per-block Prometheus metrics. ([\#6491](#6491)) - Fix small performance regression for sending invites. ([\#6493](#6493)) - Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](#6494)) Improved Documentation ---------------------- - Update documentation and variables in user contributed systemd reference file. ([\#6369](#6369), [\#6490](#6490)) - Fix link in the user directory documentation. ([\#6388](#6388)) - Add build instructions to the docker readme. ([\#6390](#6390)) - Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](#6443)) - Write some docs for the quarantine_media api. ([\#6458](#6458)) - Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](#6461)) Deprecations and Removals ------------------------- - Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](#5925)) - Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](#6488)) Internal Changes ---------------- - Add benchmarks for structured logging and improve output performance. ([\#6266](#6266)) - Improve the performance of outputting structured logging. ([\#6322](#6322)) - Refactor some code in the event authentication path for clarity. ([\#6343](#6343), [\#6468](#6468), [\#6480](#6480)) - Clean up some unnecessary quotation marks around the codebase. ([\#6362](#6362)) - Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](#6379)) - Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](#6392)) - Clarifications for the email configuration settings. ([\#6423](#6423)) - Add more tests to the blacklist when running in worker mode. ([\#6429](#6429)) - Refactor data store layer to support multiple databases in the future. ([\#6454](#6454), [\#6464](#6464), [\#6469](#6469), [\#6487](#6487)) - Port synapse.rest.client.v1 to async/await. ([\#6482](#6482)) - Port synapse.rest.client.v2_alpha to async/await. ([\#6483](#6483)) - Port SyncHandler to async/await. ([\#6484](#6484))

Synapse 1.7.0 (2019-12-13) ========================== This release changes the default settings so that only local authenticated users can query the server's room directory. See the [upgrade notes](UPGRADE.rst#upgrading-to-v170) for details. Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11. Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available [here](docs/postgres.md). A future release of synapse will, by default, disable federation for servers using SQLite. No significant changes since 1.7.0rc2. Synapse 1.7.0rc2 (2019-12-11) ============================= Bugfixes -------- - Fix incorrect error message for invalid requests when setting user's avatar URL. ([\#6497](#6497)) - Fix support for SQLite 3.7. ([\#6499](#6499)) - Fix regression where sending email push would not work when using a pusher worker. ([\#6507](#6507), [\#6509](#6509)) Synapse 1.7.0rc1 (2019-12-09) ============================= Features -------- - Implement per-room message retention policies. ([\#5815](#5815), [\#6436](#6436)) - Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](#5858)) - Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](#5925)) - Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](#6119)) - Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](#6176)) - Configure privacy-preserving settings by default for the room directory. ([\#6355](#6355)) - Add ephemeral messages support by partially implementing [MSC2228](matrix-org/matrix-spec-proposals#2228). ([\#6409](#6409)) - Add support for [MSC 2367](matrix-org/matrix-spec-proposals#2367), which allows specifying a reason on all membership events. ([\#6434](#6434)) Bugfixes -------- - Transfer non-standard power levels on room upgrade. ([\#6237](#6237)) - Fix error from the Pillow library when uploading RGBA images. ([\#6241](#6241)) - Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](#6329)) - Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](#6332)) - Prevent account data syncs getting lost across TCP replication. ([\#6333](#6333)) - Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](#6406)) - Fix an intermittent exception when handling read-receipts. ([\#6408](#6408)) - Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](#6420)) - Fix startup error when http proxy is defined. ([\#6421](#6421)) - Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](#6449)) - Fix uploading multiple cross signing signatures for the same user. ([\#6451](#6451)) - Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](#6462)) - Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](#6470)) - Improve sanity-checking when receiving events over federation. ([\#6472](#6472)) - Fix inaccurate per-block Prometheus metrics. ([\#6491](#6491)) - Fix small performance regression for sending invites. ([\#6493](#6493)) - Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](#6494)) Improved Documentation ---------------------- - Update documentation and variables in user contributed systemd reference file. ([\#6369](#6369), [\#6490](#6490)) - Fix link in the user directory documentation. ([\#6388](#6388)) - Add build instructions to the docker readme. ([\#6390](#6390)) - Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](#6443)) - Write some docs for the quarantine_media api. ([\#6458](#6458)) - Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](#6461)) Deprecations and Removals ------------------------- - Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](#5925)) - Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](#6488)) Internal Changes ---------------- - Add benchmarks for structured logging and improve output performance. ([\#6266](#6266)) - Improve the performance of outputting structured logging. ([\#6322](#6322)) - Refactor some code in the event authentication path for clarity. ([\#6343](#6343), [\#6468](#6468), [\#6480](#6480)) - Clean up some unnecessary quotation marks around the codebase. ([\#6362](#6362)) - Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](#6379)) - Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](#6392)) - Clarifications for the email configuration settings. ([\#6423](#6423)) - Add more tests to the blacklist when running in worker mode. ([\#6429](#6429)) - Refactor data store layer to support multiple databases in the future. ([\#6454](#6454), [\#6464](#6464), [\#6469](#6469), [\#6487](#6487)) - Port synapse.rest.client.v1 to async/await. ([\#6482](#6482)) - Port synapse.rest.client.v2_alpha to async/await. ([\#6483](#6483)) - Port SyncHandler to async/await. ([\#6484](#6484))

* commit '0f87b912a': Implementation of MSC2314 (#6176)

MSC2314 has now been closed, so we're backing out its implementation, which originally happened in #6176. Unfortunately it's not a direct revert, as that PR mixed in a bunch of unrelated changes to tests etc.

hawkowl added 6 commits October 7, 2019 16:20

implement v2 of state query and add tests

2c56ba0

changelog

9a9bcc4

fix

3cb804f

fix tests

8705272

fix tests

65754ed

fix tests

46ec3a0

hawkowl self-assigned this Oct 7, 2019

hawkowl requested a review from a team October 7, 2019 16:52

hawkowl added 2 commits October 8, 2019 09:27

tests dont send federation

0c8be5e

Merge branch 'develop' into hawkowl/msc2314

d4e7a25

erikjohnston reviewed Oct 8, 2019

View reviewed changes

hawkowl added 2 commits October 8, 2019 12:02

override the configs

924a699

Merge branch 'hawkowl/msc2314' of github.com:matrix-org/synapse into …

3cdf979

…hawkowl/msc2314

hawkowl requested a review from a team October 11, 2019 09:45

hawkowl added 3 commits October 21, 2019 16:14

add feature flags for msc2314 and room complexity

1263306

Merge remote-tracking branch 'origin/develop' into hawkowl/msc2314

f49cf4a

update

6ced1b2

hawkowl mentioned this pull request Oct 21, 2019

MSC2314: A Method to Backfill Room State matrix-org/matrix-spec-proposals#2314

Closed

hawkowl added 2 commits October 21, 2019 16:55

update to prefixed

380e018

remove this

efb4da5

hawkowl requested a review from erikjohnston October 28, 2019 22:07

richvdh reviewed Oct 29, 2019

View reviewed changes

hawkowl added 7 commits October 31, 2019 21:07

fix

f0b3548

fix

dfd2102

Merge remote-tracking branch 'origin/develop' into hawkowl/msc2314

e6c59cd

fix test

e7a71df

Merge remote-tracking branch 'origin/develop' into hawkowl/msc2314

4eaa010

Merge remote-tracking branch 'origin/develop' into hawkowl/msc2314

f591e7f

add sytest blacklist

e1189aa

hawkowl requested a review from a team November 14, 2019 11:21

richvdh approved these changes Nov 27, 2019

View reviewed changes

hawkowl added 2 commits November 28, 2019 06:21

Merge remote-tracking branch 'origin/develop' into hawkowl/msc2314

b48fd23

fix sytests

9e2b1e5

hawkowl merged commit 0f87b91 into develop Nov 27, 2019

hawkowl deleted the hawkowl/msc2314 branch November 27, 2019 21:54

richvdh mentioned this pull request Oct 7, 2020

add a way to tell synapse to recover a room, after a database disaster #8487

Open

babolivier pushed a commit that referenced this pull request Sep 1, 2021

Implementation of MSC2314 (#6176)

d4ac275

* commit '0f87b912a': Implementation of MSC2314 (#6176)

richvdh mentioned this pull request Apr 14, 2022

Back out implementation of MSC2314 #12474

Merged

matrixbot mentioned this pull request Dec 21, 2023

add a way to tell synapse to recover a room, after a database disaster element-hq/synapse#8487

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementation of MSC2314 #6176

Implementation of MSC2314 #6176

hawkowl commented Oct 7, 2019 •

edited

Loading

erikjohnston left a comment

richvdh left a comment

richvdh left a comment

richvdh Nov 27, 2019

richvdh Nov 27, 2019

hawkowl Nov 27, 2019

richvdh Nov 27, 2019

Implementation of MSC2314 #6176

Implementation of MSC2314 #6176

Conversation

hawkowl commented Oct 7, 2019 • edited Loading

erikjohnston left a comment

Choose a reason for hiding this comment

richvdh left a comment

Choose a reason for hiding this comment

richvdh left a comment

Choose a reason for hiding this comment

richvdh Nov 27, 2019

Choose a reason for hiding this comment

richvdh Nov 27, 2019

Choose a reason for hiding this comment

hawkowl Nov 27, 2019

Choose a reason for hiding this comment

richvdh Nov 27, 2019

Choose a reason for hiding this comment

hawkowl commented Oct 7, 2019 •

edited

Loading