fix(library): fix PlaylistDAO::removeHiddenTracks performance
#11851
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The query consists of an outer query and inner subquery. In the previous form, the inner query had a dependency on `p1.playlist_id` of the outer query. Which could change for each row of the outer select statement. This made the inner query a `CORRELATED LIST SUBQUERY`, which caused the subquery to be re-evaluated for each row of the outer query. This dependency however is unnecessary because of the `AND p1.playlist_id=:id` restriction in the outer query. So in theory, `p1.playlist_id` never changes. Unfortunately, the SQLite was not able to make that deduction which lead to the dependency. Manually specifying that the `p1.playlist_id` is constant for the entire subquery, removes the dependecy to the outer query, resulting in a plain `LIST SUBQUERY` which can be evaluated once and then reused for all rows of the outer subquery. This results in reduced algorithmic complexity and thus drastically improved performance for large tables.
|
Just to get an idea of the impact of this tiny fix:
🎉 As mentioned in #11724 (comment) I can't tell whether that query actually does what it should since the playlists seem to be cleaned up during startup already (and i don't rescan the collection on startup). |
ronso0
approved these changes
Aug 21, 2023
|
LGTM, thanks a lot! |
The effect of algorithmic complexity in action ;)
Thank even more for testing and figuring out what line and commit caused the regression. That cut the time required ten fold. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes #11724
The query consists of an outer query and inner subquery. In the previous form, the inner query had a dependency on
p1.playlist_idof the outer query. Which could change for each row of the outer select statement. This made the inner query aCORRELATED LIST SUBQUERY, which caused the subquery to be re-evaluated for each row of the outer query. This dependency however is unnecessary because of theAND p1.playlist_id=:idrestriction in the outer query. So in theory,p1.playlist_idnever changes. Unfortunately, the SQLite was not able to make that deduction which lead to the dependency. Manually specifying that thep1.playlist_idis constant for the entire subquery, removes the dependecy to the outer query, resulting in a plainLIST SUBQUERYwhich can be evaluated once and then reused for all rows of the outer subquery. This results in reduced algorithmic complexity and thus drastically improved performance for large tables.