Remove any NULL characters from remote displaynames before updating user directory #12743
Conversation
|
Ahh, I see: this is specific to the user directory. |
DMRobertson
changed the title
synapse/handlers/user_directory.py
Outdated
| # Replace any NULL characters in the name as these cannot be stored in the database | ||
| new_name = new_name.replace("\x00", "\uFFFD") |
There was a problem hiding this comment.
For completeness, sqlite sort-of-supports null-codepointsin strings, with scary caveats: https://sqlite.org/nulinstr.html
Postgres definitely doesn't and won't any time soon. See e.g. this HN post.
synapse/handlers/user_directory.py
Outdated
| @@ -464,6 +464,10 @@ async def _handle_possible_remote_profile_change( | |||
|
|
|||
| prev_name = prev_event.content.get("displayname") | |||
| new_name = event.content.get("displayname") | |||
|
|
|||
| # Replace any NULL characters in the name as these cannot be stored in the database | |||
| new_name = new_name.replace("\x00", "\uFFFD") | |||
There was a problem hiding this comment.
We used a space as the replacement character in #10820. I think this might have been chosen to make sure that the text-search stuff (as in https://www.postgresql.org/docs/current/datatype-textsearch.html and https://www.postgresql.org/docs/current/textsearch-controls.html) work more nicely. Let me see if I can experiment to see how postgres handles a replacement character in that context...
synapse/handlers/user_directory.py
Outdated
| # Replace any NULL characters in the name as these cannot be stored in the database | ||
| new_name = new_name.replace("\x00", "\uFFFD") |
There was a problem hiding this comment.
We'll want to move this after the isinstance(new_name, str) validation below.
... actually do we want to do it after the has-anything-changed condition?:
prev_name != new_name or prev_avatar != new_avatar
|
I think I'd put the check in |
Extract the shared function into storage utils and reuse in both locations for consistent results.
An event that looks like this:
{ "content": { "displayname": "\u0001VERSION\u0001\u0000", "membership": "join" } }Causes an exception updating the user directory table
A string literal cannot contain NUL (0x00) characterswhich never passes and just keeps retrying consuming a lot of CPU on main.Is this the right fix?
Signed off by Nick ([email protected])
Pull Request Checklist
(run the linters)