MSC2881: Message Attachments

[MurzNN]

Rendered

Depends on MSC3051: A scalable relation format

Related issues:

• Attach multiple images (files) to message
• Show albums for multiple images
• Message attachments (multiple photos, files, other messages for forwarding, etc)
• [Redesign: timeline: implement new file transfer UX](Redesign: timeline: implement new file transfer UX element-hq/element-web#7565
• Image galleries & captions element-hq/roadmap#13

Also lookup alternative and more optimal implementation: #3382

Signed-off-by: Alexey Korepov [email protected]

[MurzNN]

The Hummingbard project seems use "Option two" implementation from this MSC for attaching multiple files/images/links to event! 🎉

[kevincox]

What happens if such an event is never referenced? Does it just stay hidden forever? This is probably the best option but it is worth discussing.

Pros:

• Allows uploading the media as the user composes so that sending is quicker.
• Messages don't suddenly appear (for example) a day later higher up in the timeline.
• If the sender crashes or fails it can just restart. It isn't required to recollect the event IDs and reuse them. (Although reuse would be more efficient)

Cons:

• Invisible events may be an abuse vector. (DoS or covert channel)
• Upon a failed send or deleted message (if the client eagerly uploads media) the user likely doesn't realize that the media was uploaded and available in the room history. This may be a concern if the user then realized that they didn't want to send that info. (Maybe they tried to cancel the send because it was in the wrong room and would have redacted the media if they realized it was actually sent).

So basically it is nice because it emulates an atomic commit. But it is not nice because it does expose some side effects even if not committed.

[kevincox]

For a large number of attachments (for example "look at thses photos from my two week vacation!") many clients won't want to show all of the images upfront. Most would probably render some sort of grid with the first couple then a "37 more" link.

In order to help with cross-client consistency should we specify something about the order and how that should be handled? You do mention "rearranging" attachments earlier.

For example:

• Clients SHOULD show the attachments in the order the appear in the m.relates_to array.
• If the client decides to show only a subset of photos upfront it SHOULD prefer to show the ones earliest in the m.relates_to array.

I think it should be non-normative, but basically if you are going to only show some show the first ones rather than the last ones.

[kevincox]

I don't think we need to specify "must" here. For example if the image is bigger than the users display I think most users would rather have it scaled to fit rather than needing to scroll around. I would leave this section as an example of a possible implementation.

[kevincox]

What happens if such an event is never referenced? Does it just stay hidden forever? This is probably the best option but it is worth discussing.

Pros:

• Allows uploading the media as the user composes so that sending is quicker.
• Messages don't suddenly appear (for example) a day later higher up in the timeline.
• If the sender crashes or fails it can just restart. It isn't required to recollect the event IDs and reuse them. (Although reuse would be more efficient)

Cons:

• Invisible events may be an abuse vector. (DoS or covert channel)
• Upon a failed send or deleted message (if the client eagerly uploads media) the user likely doesn't realize that the media was uploaded and available in the room history. This may be a concern if the user then realizes that they didn't want to send that info. (Maybe they tried to cancel the send because it was in the wrong room and would have redacted the media if they realized it was actually sent).

So basically it is nice because it emulates an atomic commit. But it is not nice because it does expose some side effects even if not committed.

[chayleaf]

there is already many ways to send an invisible event, that by itself shouldn't be considered an issue. The other points are still valid though

[kevincox]

I think this should be put into the MSC proper. I think it makes sense that redacting an attachment implies that the attachment is removed from the messages that include it. However I would leave it open to clients showing a "redacted" placeholder or just hiding it outright. (For example if an attachment was referenced from multiple events it may be helpful to see which events now have missing attachments).

[chayleaf]

Currently Synapse doesn't allow m.relates_to to be an array, so the first option isn't doable as-is, but it's possible to implement with a limited attachment count (1). That also won't allow to reply with images.

[MurzNN]

Currently Synapse doesn't allow m.relates_to to be an array, so the first option isn't doable as-is, but it's possible to implement with a limited attachment count (1). That also won't allow to reply with images.

@pavlukivan Thanks for identifying the problem, m.relates_to is object, not array, and now already late to change it. What do you think about using the separate key in object for attachments? Something like this:

{
  "type": "m.room.message",
  "content": {
    "msgtype": "m.text",
    "body": "Here is my photos and videos from yesterday event",
    "m.relates_to": [
      {
        "m.in_reply_to": {
          "event_id": "$id_of_replied_message"
        },
        "attachments": {
          "event_ids": [
            "$id_of_previosly_send_media_event_1",
            "$id_of_previosly_send_media_event_2",
            "$id_of_previosly_send_media_event_3",
          ]
        }
      },
      {
        "rel_type": "m.attachment",
        "event_id": "$id_of_previosly_send_media_event_2"
      }
    ]
  }
}

Or even we can move attachments to upper level, but, as I guess, Synapse must implement some optimizations to deliver events, mentioned in m.relates_to together with main event (or at least preload them for quicker access).

[chayleaf]

What do you think about using the separate key in object for attachments?

While it seems like a hack, I guess it will offer the best compatibility (old clients won't be required to handle array relations). Of course I'd prefer the option where relations become an array (or similar), but it would require client support to understand the new format at all.

[MurzNN]

While it seems like a hack, I guess it will offer the best compatibility (old clients won't be required to handle array relations). Of course I'd prefer the option where relations become an array (or similar), but it would require client support to understand the new format at all.

I suppose that implementing multiple attachments without array as storage is impossible, and implementing with limitation of only one attachment is useless! Thus we still need to insert the array somewhere :)

And the best way, I think, is to implement some type of arrays in MSC1767: Extensible events in Matrix (and follow it in current MSC) - here is my comment about this #1767 (comment)

[deepbluev7]

Thanks for identifying the problem, m.relates_to is object, not array, and now already late to change it.

Not really. #2674 hasn't landed yet and one could use #3051 instead. But so far there seem to be not enough use cases to allow multiple relations.

[chayleaf]

I think using an array still makes sense, considering it adds very little overhead but adds use cases like this. The only downside I see is it being harder to store relations in the database, as one-to-many is considerably easier to store.

[kevincox]

This seems like a clear answer of "we will need it". It is easy to imagine and we have a real-world use case here.

[chayleaf]

But so far there seem to be not enough use cases to allow multiple relations.

Off the top of my head: replying to multiple message at the same time, or starting a thread from multiple messages could be done with that

[turt2live]

@MurzNN I'm not sure what happened here, but the diff is showing quite a few changes. I would recommend either fixing the branch, or, if preferred, closing this PR and opening a new MSC off a dedicated branch.

[richvdh]

suspect this is the problem:

see #3367

[MurzNN]

@richvdh Thanks for the tip, I've switched to main branch, seems now all is well.