room stays unread if no messages due to retention policy

[4nd3r]

Initial report here: element-hq/element-web#14976

Description

If older messages gets removed due to retention policies and no new messages after retention time, room becomes unread forever until new message is inserted.

Tested clients: Element (web, Android and iOS)

Steps to reproduce

• make new room
• set retention policy to short time
• send message and wait until retention kicks in
• clear client's cache and reload
• room stays unread

Version information

• Homeserver: kvlt.ee

• Version: {"server_version":"1.41.1","python_version":"3.7.3"}

• Install method: pip

• Platform: Debian Buster

[4nd3r]

Still present in 1.48.

If I have to vote, then it's the most annoying bug in Synapse and I wouldn't say it's tolerable - please reconsider this classification. From user's pespective it's confusing and I have to bump dozens of chats with gibberish just to get state to read.

Maybe worth resurrecting element-hq/element-web#14976 and fix it client side, because something is different with Element for iOS, because there is no such issue...

It seems to me that fixing this issue has low priority until matrix-org/matrix-spec-proposals#1763 lands, right? I would really love to see retention related issues as high priority, because it's unthinkable in the year 2021, that you have to store message and media (#6832) history forever.

[4nd3r]

Discovered a way to set room as read.

1. Open /devtools (in element-web).
2. Explore Room State -> m.room.retention -> Edit.
3. Remember current values of min_lifetime and max_lifetime.
4. Set min_lifetime and max_lifetime to 0 and press Send.
5. Repeat same action again with memorized values.
6. Room becomes unread.

[DanceMore]

If I have to vote, then it's the most annoying bug in Synapse and I wouldn't say it's tolerable - please reconsider this classification. From user's pespective it's confusing and I have to bump dozens of chats with gibberish just to get state to read.

I think it gets worse if you have multiple sessions open or sign in on another device. I have wildly inconsistent unread messages across multiple devices.

basically only my "oldest session" looks anything close to correct. new logins wind up having perpetually-unread direct-messages as well as perpetually-unread channels for all but the most-active rooms.

the feature's documentation and implementation sure look pretty close to complete. I also think it's almost a "killer-feature", both from an admin and user perspective. it would be a shame for it to languish in obscurity as "technically experimental".

[SPiRiT369]

I would like to add something.
I am not sure if this is a client problem (Element) or something with Synapse, but anyway - room with retention policy that has its first "unread" message deleted, will make many calls to /read_markers (when users hover on each message), causing many un-ending "404 Event Id not found in the room" calls. That's bad. I'm not sure if it's the client side who should handle such cases or the server to provide a different response, just adding it here so people know. (IMHO should get a bit more priority)

[clokep]

Note that retention policies are an experimental features and there are some rough edges.

[MurzNN]

Does anybody invented any manual workaround to fix "unread" status of the room if all messages is deleted from room? Something like remove read_marker?

Regarding to spec, m.fully_read is required for /_matrix/client/v3/rooms/{roomId}/read_markers, but maybe it can have an empty content? If this is not allowed by spec, maybe make a fix in Synapse that allow this as workaround?

[MurzNN]

Now in rooms with such problem Synapse on GET/_matrix/client/r0/rooms/{roomId}/read_markers returns a 404 error with content:

{errcode: "M_UNKNOWN", error: "Could not find event $fOKaObYgyIi7AZerFASXqZGQrdi-KgBwsDfNB_PPZyA"}

And workaround from Synapse side can be like this:
If Synapse catch the missing event problem, before returning 404 error do make count of total events in room, and if it equals zero - reset read_marker to some last state event.

What do you think about this?

[rettichschnidi]

Does anybody invented any manual workaround to fix "unread" status of the room if all messages is deleted from room?

Not elegant at all, but for DMs, I just leave the rooms. :/

[Sirius-KiH]

We have the same problem on our sever. When trying to make a request to read_markers, element logs hundreds of error messages to console:

matrix.redacted.com/_matrix/client/r0/rooms/!XXXXXXXXXXX%3Aredacted.com/read_markers:1  Failed to load resource: the server responded with a status of 404 ()

Is there still no workaround? Unfortunately this is really starting to impact usability for our users, and disabling retention is also not an option due to privacy reasons.

[cristianyxen]

I fixed the issue on my server with this cristianyxen@d5aed2c

[Takalele]

@cristianyxen,

thank you, your fix fixed it also for my homeserver.
Are you considering to create a pull request to push your fix upstream?

BR
Takalele

[cristianyxen]

thank you, your fix fixed it also for my homeserver. Are you considering to create a pull request to push your fix upstream?

Well I could, but I think it would be better to fix the underlaying issue that seems to be in simple_select_one() not handling cases where no data is available.