This repository has been archived by the owner on Apr 26, 2024. It is now read-only.
Refactor store.have_events #3117
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
richvdh
force-pushed
the
rav/refactor_have_events
branch
from
a6c5661 to
5d550fc
Compare
erikjohnston
suggested changes
Apr 18, 2018
| @@ -394,7 +394,7 @@ def get_events(self, destinations, room_id, event_ids, return_local=True): | |||
| seen_events = yield self.store.get_events(event_ids, allow_rejected=True) | |||
| signed_events = seen_events.values() | |||
| else: | |||
| seen_events = yield self.store.have_events(event_ids) | |||
| seen_events = yield self.store.get_rejection_reasons(event_ids) | |||
There was a problem hiding this comment.
Shouldn't this be have_seen_events?
synapse/storage/events.py
Outdated
| ) | ||
| defer.returnValue(results) | ||
|
|
||
| def get_rejection_reasons(self, event_ids): |
There was a problem hiding this comment.
Reading through this PR I've been a bit confused about this name. I keep expecting it to just return the rejected events, rather than all events we've seen. Maybe something like get_seen_events_with_rejections? Its horrific but I suck at naming :/
| # break the input up into chunks of 100 | ||
| input_iterator = iter(event_ids) | ||
| for chunk in iter(lambda: list(itertools.islice(input_iterator, 100)), | ||
| []): |
There was a problem hiding this comment.
We tend to do:
chunks = [
iterable[i:i + batch_size]
for i in xrange(0, len(iterable), batch_size)
]which I think is quite a bit clearer?
There was a problem hiding this comment.
clearer, but doesn't actually work when iterable is an iterable rather than a sequence.
There was a problem hiding this comment.
(and involves copying the input)
|
retest this please |
|
ok the postgres build failed because postgres wasn't running. PTAL @erikjohnston |
erikjohnston
approved these changes
Apr 19, 2018
It turns out that most of the time we were calling have_events, we were only using half of the result. Replace have_events with have_seen_events and get_rejection_reasons, so that we can see what's going on a bit more clearly.
richvdh
force-pushed
the
rav/refactor_have_events
branch
from
61e1c35 to
b1dfbc3
Compare
neilisfragile
added a commit
that referenced
this pull request
Apr 27, 2018
Changes in synapse v0.28.0-rc1 (2018-04-26) =========================================== Bug Fixes: * Fix quarantine media admin API and search reindex (PR #3130) * Fix media admin APIs (PR #3134) Changes in synapse v0.28.0-rc1 (2018-04-24) =========================================== Minor performance improvement to federation sending and bug fixes. (Note: This release does not include state resolutions discussed in matrix live) Features: * Add metrics for event processing lag (PR #3090) * Add metrics for ResponseCache (PR #3092) Changes: * Synapse on PyPy (PR #2760) Thanks to @Valodim! * move handling of auto_join_rooms to RegisterHandler (PR #2996) Thanks to @krombel! * Improve handling of SRV records for federation connections (PR #3016) Thanks to @silkeh! * Document the behaviour of ResponseCache (PR #3059) * Preparation for py3 (PR #3061, #3073, #3074, #3075, #3103, #3104, #3106, #3107, #3109, #3110) Thanks to @NotAFile! * update prometheus dashboard to use new metric names (PR #3069) Thanks to @krombel! * use python3-compatible prints (PR #3074) Thanks to @NotAFile! * Send federation events concurrently (PR #3078) * Limit concurrent event sends for a room (PR #3079) * Improve R30 stat definition (PR #3086) * Send events to ASes concurrently (PR #3088) * Refactor ResponseCache usage (PR #3093) * Clarify that SRV may not point to a CNAME (PR #3100) Thanks to @silkeh! * Use str(e) instead of e.message (PR #3103) Thanks to @NotAFile! * Use six.itervalues in some places (PR #3106) Thanks to @NotAFile! * Refactor store.have_events (PR #3117) Bug Fixes: * Return 401 for invalid access_token on logout (PR #2938) Thanks to @dklug! * Return a 404 rather than a 500 on rejoining empty rooms (PR #3080) * fix federation_domain_whitelist (PR #3099) * Avoid creating events with huge numbers of prev_events (PR #3113) * Reject events which have lots of prev_events (PR #3118)
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It turns out that most of the time we were calling have_events, we were only
using half of the result. Replace have_events with have_seen_events and
get_rejection_reasons, so that we can see what's going on a bit more clearly.