-
-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SQLite: Replace LIKE with SUBSTR for file path matching #4162
Conversation
Pull Request Test Coverage Report for Build 1105672637
💛 - Coveralls |
Are you sure this does not affect 2.3? |
As I mentioned somehow it does, but no one complained so far. I won't fix it, let's move forward. |
I don't want to apply this to 2.3, too risky. We cannot fix all the issues for each and every version. Hopefully we can get rid of the legacy database code and Qt UTF-16 strings someday. |
The database code is completely broken because the LIKE operator is tweaked unconditionally for searching, i.e. by treating 'ä' like 'a'. But we also used it for exact file path prefix matches which obviously doesn't work as intended and could cause false positives. |
The modified test even succeeds for 2.3. |
src/library/dao/directorydao.cpp
Outdated
"track_locations.location LIKE :startsWithOldDirectory ESCAPE :escape")); | ||
query.bindValue(":startsWithOldDirectory", startsWithOldDirectory); | ||
query.bindValue(":escape", kSqlLikeMatchAll); | ||
"SUBSTR(track_locations.location,1,:oldDirectoryPrefixLength)=:oldDirectoryPrefix")); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We usually don't explain the obvious in comments, but in this case it might be good to add a comment that 1
refers to the first character in the string (i.e. it's not 0-based like arrays/lists/whatever in C/C++ are).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another way to implement something like STARTSWITH()
would be
INSTR(track_locations.location, :oldDirectoryPrefix) = 1
but I guess that would be less performant.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I won't replicate the same comment at 3 different locations. This can be looked up in the SQLite docs.
Using INSTR is less chatty, we should probably use that!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The naming of the variables clearly reveals the purpose, so there is no need to explain the 1-based indexing.
Thinking about the performance implications SUBSTR might in fact be faster than INSTR, because you have to know the outer context for optimizing the INSTR operation. I will keep using it in the aoide backend.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thank you!
This refactoring makes SqlLikeWildcardEscaper obsolete and ensures that file paths in the database are matched case-sensitive.
Follow-up of #4146. It also fixes the incorrect file path matching in TrackDAO. No backport to 2.3.
Note: We don't properly escape SQL LIKE wildcards in other, informal search queries, e.g. for crates. But this is just a minor issue and if SqlLikeWildcardEscaper is ever needed again it could be brought back. Currently it is unused and should disappear.