Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Speed up deleting device messages #16643

Merged
merged 7 commits into from
Nov 16, 2023

Conversation

erikjohnston
Copy link
Member

Keeping track of a lower bound of stream ID where we've deleted everything below makes the queries much faster. Otherwise, every time we scan for rows to delete we'd re-scan across all the rows that have previously deleted (until the next table VACUUM).

@erikjohnston erikjohnston force-pushed the erikj/faster_device_message_deletion branch from 67db2de to df6ab79 Compare November 16, 2023 10:51
@erikjohnston erikjohnston force-pushed the erikj/faster_device_message_deletion branch from df6ab79 to 1293012 Compare November 16, 2023 10:57
@erikjohnston erikjohnston marked this pull request as ready for review November 16, 2023 11:06
@erikjohnston erikjohnston requested a review from a team as a code owner November 16, 2023 11:06
Copy link
Contributor

@reivilibre reivilibre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That (avoiding looking at the dead tuples) is clever and so obvious in hindsight!

synapse/storage/databases/main/deviceinbox.py Outdated Show resolved Hide resolved
@erikjohnston erikjohnston requested a review from clokep November 16, 2023 12:54
@erikjohnston erikjohnston requested a review from clokep November 16, 2023 14:27
@erikjohnston erikjohnston merged commit 3e8531d into develop Nov 16, 2023
39 of 41 checks passed
@erikjohnston erikjohnston deleted the erikj/faster_device_message_deletion branch November 16, 2023 15:19
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants