-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Conversation
as per the proposal; we can deduplicate redundant lazy-loaded members which are sent in the same sync sequence. we do this heuristically rather than requiring the client to somehow tell us which members it has chosen to cache, by instead caching the last N members sent to a client, and not sending them again. For now we hardcode N to 100. Each cache for a given (user,device) tuple is in turn cached for up to X minutes (to avoid the caches building up). For now we hardcode X to 30.
works, although considers all room membership events rather than just joined ones as i'm failing to see a nice way to filter based on membership
synapse/handlers/sync.py
Outdated
|
||
missing_hero_event_ids = [ | ||
member_ids[hero_id] | ||
for hero_id in summary['heros'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be m.heros
? I get a exceptions.KeyError: 'heros'
when running this locally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed it, as am 99% certain that that is what it should be, and it unblocked me locally from syncing.
|
Yup, see #3574 (comment). There are a bunch of correctness errors here, but at least the API can be developed against even if the data it gives is a bit wrong currently. See also the other FIXMEs at https://github.com/matrix-org/synapse/pull/3574/files#diff-6fee1701e980dbc9da57a5a108ee7b65R556 for other limitations right now. |
Yeah, just realized you mentioned this in the description |
@richvdh PTAL. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm modulo nitpicking about early returns.
Note that the PR is currently based on #3568 which has pending comments.
synapse/handlers/sync.py
Outdated
summary["m.joined_member_count"] = len(joined_user_ids) | ||
summary["m.invited_member_count"] = len(invited_user_ids) | ||
|
||
if not name_id and not canonical_alias_id: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we do an early return if name_id or canonical_alias_id
rather than having this massive if
block?
synapse/handlers/sync.py
Outdated
[user_id for user_id in member_ids.keys() if user_id != me] | ||
)[0:5] | ||
|
||
if sync_config.filter_collection.lazy_load_members(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again, make this into an early return?
I'm very dubious about the early returns; it feels much less error prone to say "only do this thing if this condition is true" rather than "return early if this condition is false, as coincidentally the remainder of the method is handling that condition being true" - as:
However, I've gone and done it anyway in a bid to get this merged, but i am still wondering whether we are making reviews too subjective (or making it too hard to determine the 'strength' of review feedback - and whether it's a "if i were doing this, i'd have considered doing it like this, but it's pretty subjective" or a "this is cosmetic nitpick but something that Should Be Fixed.") |
@richvdh ptal |
These are no longer relevant as the changes were in 0.33.2.
I already LGTMed this; it now looks even better. Just waiting for the tests to run. To address your points, certainly a better alternative might be to split out separate functions.
yes it does; I don't really see that as a problem.
Personally I find early returns much easier to read than having to scroll up and down to figure out how the indentation matches up and which "if" clause we're leaving.
I think adding stuff to the end of a function without looking at what else is going on would be, er, foolhardy.
sure it is.
well, we discussed this in the team meeting the other day, and yes, these topics were discussed. I don't think here is the right place to reiterate that conversation, but in this case: I am coming to this as the person who is probably going to have to maintain this code (or at least review future changes to it in the future), and I do not want to have to maintain code that has massive nested if blocks that are hard to follow. Thank you for changing it. |
synapse/handlers/sync.py
Outdated
@@ -626,6 +626,16 @@ def compute_summary(self, room_id, sync_config, batch, state, now_token): | |||
|
|||
defer.returnValue(summary) | |||
|
|||
def get_lazy_loaded_members_cache(self, cache_key): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: in future, please do add docstrings to public functions: in particular it's important to document the args and return types.
Features -------- - Add support for the SNI extension to federation TLS connections. Thanks to @vojeroen! ([\#3439](#3439)) - Add /_media/r0/config ([\#3184](#3184)) - speed up /members API and add `at` and `membership` params as per MSC1227 ([\#3568](#3568)) - implement `summary` block in /sync response as per MSC688 ([\#3574](#3574)) - Add lazy-loading support to /messages as per MSC1227 ([\#3589](#3589)) - Add ability to limit number of monthly active users on the server ([\#3633](#3633)) - Support more federation endpoints on workers ([\#3653](#3653)) - Basic support for room versioning ([\#3654](#3654)) - Ability to disable client/server Synapse via conf toggle ([\#3655](#3655)) - Ability to whitelist specific threepids against monthly active user limiting ([\#3662](#3662)) - Add some metrics for the appservice and federation event sending loops ([\#3664](#3664)) - Where server is disabled, block ability for locked out users to read new messages ([\#3670](#3670)) - set admin uri via config, to be used in error messages where the user should contact the administrator ([\#3687](#3687)) - Synapse's presence functionality can now be disabled with the "use_presence" configuration option. ([\#3694](#3694)) - For resource limit blocked users, prevent writing into rooms ([\#3708](#3708)) Bugfixes -------- - Fix occasional glitches in the synapse_event_persisted_position metric ([\#3658](#3658)) - Fix bug on deleting 3pid when using identity servers that don't support unbind API ([\#3661](#3661)) - Make the tests pass on Twisted < 18.7.0 ([\#3676](#3676)) - Don’t ship recaptcha_ajax.js, use it directly from Google ([\#3677](#3677)) - Fixes test_reap_monthly_active_users so it passes under postgres ([\#3681](#3681)) - Fix mau blocking calulation bug on login ([\#3689](#3689)) - Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users ([\#3692](#3692)) - Improve HTTP request logging to include all requests ([\#3700](#3700)) - Avoid timing out requests while we are streaming back the response ([\#3701](#3701)) - Support more federation endpoints on workers ([\#3705](#3705), [\#3713](#3713)) - Fix "Starting db txn 'get_all_updated_receipts' from sentinel context" warning ([\#3710](#3710)) - Fix bug where `state_cache` cache factor ignored environment variables ([\#3719](#3719)) - Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](#3723)) - Fix bug introduced in v0.33.3rc1 which made the ToS give a 500 error ([\#3732](#3732)) Deprecations and Removals ------------------------- - The Shared-Secret registration method of the legacy v1/register REST endpoint has been removed. For a replacement, please see [the admin/register API documentation](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/register_api.rst). ([\#3703](#3703)) Internal Changes ---------------- - The test suite now can run under PostgreSQL. ([\#3423](#3423)) - Refactor HTTP replication endpoints to reduce code duplication ([\#3632](#3632)) - Tests now correctly execute on Python 3. ([\#3647](#3647)) - Sytests can now be run inside a Docker container. ([\#3660](#3660)) - Port over enough to Python 3 to allow the sytests to start. ([\#3668](#3668)) - Update docker base image from alpine 3.7 to 3.8. ([\#3669](#3669)) - Rename synapse.util.async to synapse.util.async_helpers to mitigate async becoming a keyword on Python 3.7. ([\#3678](#3678)) - Synapse's tests are now formatted with the black autoformatter. ([\#3679](#3679)) - Implemented a new testing base class to reduce test boilerplate. ([\#3684](#3684)) - Rename MAU prometheus metrics ([\#3690](#3690)) - add new error type ResourceLimit ([\#3707](#3707)) - Logcontexts for replication command handlers ([\#3709](#3709)) - Update admin register API documentation to reference a real user ID. ([\#3712](#3712))
this works, although considers all room membership events rather than just joined
ones as i'm failing to see a nice way to filter based on membership.
Implements matrix-org/matrix-spec-proposals#688
Builds on #3568 (in turn #3567 (in turn #3331 (in turn #2970))).(all now merged)Sytest at matrix-org/sytest#470