Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: MSC3215: Aristotle - Moderation in all things #3215

Draft
wants to merge 12 commits into
base: old_master
Choose a base branch
from
Draft

Draft: MSC3215: Aristotle - Moderation in all things #3215

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
23159fb
MSC3215: Aristotle - Moderation in all things
Yoric May 24, 2021
87d9da4
TODO: Routing messages.
Yoric May 24, 2021
80e78e0
Let's try this with a Routing Bot
Yoric May 24, 2021
1a826ad
Discussing limitations
Yoric May 24, 2021
a379b78
More security discussions
Yoric May 25, 2021
da689ab
tweak
Yoric May 25, 2021
c941c49
anoa's comments
Yoric May 25, 2021
79d9a66
More clarification
Yoric May 25, 2021
ad08ef8
Mentioning P2P
Yoric May 25, 2021
c21243e
Canceling an abuse report
Yoric May 26, 2021
b3e1db1
Update proposals/3215-towards-decentralized-moderation.md
Yoric Jun 7, 2021
9439f62
Typo fixes
Yoric Jun 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 87 additions & 49 deletions proposals/3215-towards-decentralized-moderation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,101 +70,128 @@ can be invited to moderation rooms act upon abuse reports:

### Invariants

- Each room MAY have a state event `m.room.moderation_room`. If specified, this is the room ID towards which
abuse reports MUST be sent. As rooms may be deleted `m.room.moderation_room` MAY be an invalid room ID.
- Each room MAY have a state event `m.room.moderated_by`. If specified, this is the room ID towards which
abuse reports MUST be sent. As rooms may be deleted `m.room.moderated_by` MAY be an invalid room ID.
A room that has a state event `m.room.moderated_by` supports moderation.

```jsonc
{
"state_key": "m.room.moderation_room",
"type": "m.room.moderation_room",
"state_key": "m.room.moderated_by",
"type": "m.room.moderated_by",
"content": {
"room_id": XXX, // The room picked for moderation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fail to see the purpose of this room. IIUC users never actually use this room, they likely don't even have access.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably to couple with ban lists as rooms.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fail to see the purpose of this room. IIUC users never actually use this room, they likely don't even have access.

The entire room or specifying the room ID in the state event?

If the former, well, we need to send abuse reports somewhere. The current abuse API has them sent to a proprietary admin API. We replace this with a standard room. At this stage, it's up to users and tooling to decide what they do with it.

If the latter, the client needs a way to find where to post the abuse reports.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I am talking about in the state event.

If the latter, the client needs a way to find where to post the abuse reports.

Please elaborate, I thought they just talked to the bot.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please elaborate, I thought they just talked to the bot.

The bot is just a delivery mechanism to send a message to the Moderation Room. The same bot may be used by several Moderation Rooms. So we need both the userID of the bot (to talk to it) and the roomID of the Moderation Room (to tell it where to send the message).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why doesn't the bot have a mapping from source room to destination.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using field room_id from the m.abuse.report message to know where the room comes from?

Yes, this could work, too, if we require the bot to be stateful. I believe that the best way to do it, though, is to keep rooms themselves the source of truth, rather than some bot memory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Does this mean that the bot has to peek into the "community room" to see where it should send the report to? Or is the bot expected to be part of that room already?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the current status of the MSC, the user's client copies this value room_id as field moderated_by_id in the m.abuse.report. This lets the bot find out where to route the message without having to peek into the room.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I missed that. It still feels weird to me that we need to expose this to the user but I'll consider this resolved until I have a better idea what to do here.

"user_id": XXX, // The bot in charge of forwarding reports to `room_id`.
Copy link
Contributor

@kevincox kevincox May 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why enforce that there is a bot? This seems to be over-complicated. Why not just provide a list of users that should be invited to a report room? This could be a bot, or it could just be the room admin(s). This way the use of a bot is not mandated. This also has a number of advantages, such as maybe one of the admins is causing the issue, the user could choose to exclude that user (this benefit is lost in the case of a bot, but at least it is allowed in some cases). Furthermore the user can use E2EE if they know the admin's keys. This removes an extra set of keys that need to be dealt with.

It is also much simpler, especially for the case of a small team of admins that manage a room or two. Now they don't need to set up rooms, bots or anything. Yet they are still prepared if a (rare) abuse report comes in. In fact I would even consider that clients recommend reporting to the users of highest power-level in the room if this event is not present, this means that there is some sort of reasonable route for reporting abuse even if the room moderators haven't considered that abuse may concern. (People generally don't think about these issues until they happen)

The process then becomes:

  1. Create a room.
  2. Invite the users listed in the m.room.moderated_by event. If there is no such event invite all of the users at the highest power level in the room.
    a. Optional: The user may instead invite only a subset of this list.
  3. Send an m.abuse.report as described below in the MSC.

At that point that report may be handled by the bot listed in the m.room.moderated_by event, it may be manually forwarded to another room for internal discussion among the moderators, or it may just be discussed in the reporting room. This approach seems much simpler and much more flexible.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main reason for there being a bot is that Matrix does not offer a built-in mechanism for users who are not member of a room (in this case, the Moderation Room) to post events towards that room. The bot is the simplest routing mechanism that I can think of. If I read correctly your counter-proposal, it does not address this (rather fundamental) issue.

In fact I would even consider that clients recommend reporting to the users of highest power-level in the room if this event is not present, this means that there is some sort of reasonable route for reporting abuse even if the room moderators haven't considered that abuse may concern.

Good idea. We can definitely add this as a suggestion in the MSC if the state events are not setup.

At that point that report may be handled by the bot listed in the m.room.moderated_by event, it may be manually forwarded to another room for internal discussion among the moderators, or it may just be discussed in the reporting room. This approach seems much simpler and much more flexible.

What's the "reporting room"? If it's what I call the Community Room, receiving abuse reports in the same room as they were sent leads to immediate deanonymization of the reporters.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I read correctly your counter-proposal, it does not address this (rather fundamental) issue.

It does address it, but it does effectively sidestep it. It allows you to use a bot which forwards to a room, or it lets you just use the room that the user created to send the report. It puts the choice to the moderation team.

What's the "reporting room"?

I mean the room the reporter created with the bot.

Let me summarize the flow you are requiring in this MSC.

  1. Create a room with $bot.
  2. Send an abuse report.
  3. $bot copies report to $modroom.

That is a completely reasonable report flow. However it seems overly specified. There are many other valid moderation workflows that don't need or want this complexity. Especially if you have a smallish community abuse reports will be rare. So having a dedicated mod room for discussion is probably not necessary. And in many communities the mods may want to keep the separate reports in different rooms for organization. Also keep in mind that for many (probably most) communities the "mod team" is one person. So copying the abuse report to $modroom is quite pointless. It just lets that one mod discuss with themselves.

What I am suggesting is that we just drop everything about the bot and the moderation room from this MSC. It can still be implemented, but it leaves each mod team free to implement their own workflow and leaves room for different bots that work differently (and leaves room for no bot at all). In this case the MSC becomes

  1. Create a room with $reportusers.
  2. Send an abuse report.

This way we have a standardized method for reporting abuse. But the actual workflow for handling it is still flexible. It is totally valid to use a bot as suggested in this MSC to copy the report to a moderation room, however that is optional, not required.

This has a number of advantages in my mind.

  • Small mod teams don't need to configure a bot.
  • Small mod teams don't require any configuration.
  • Leaves a lot of room for flexibility. Maybe instead of forwarding to the "mod room" it files a ticket in a ticketing system.

TL;DR I don't see the benefit of mandating the bot and its behaviour.

Copy link
Author

@Yoric Yoric Jun 3, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does the reporter send the abuse report in your counter-proposal? To a userID as specified in moderated_by?

If I understand correctly, you're splitting the MSC in two. The bot and moderation room are still necessary in many (most?) cases but remain unspecified. Essentially, we're losing the specification for moderator_of and how it interacts with possible bots.

  • Small mod teams don't need to configure a bot.
  • Small mod teams don't require any configuration.

A few notes on this:

  1. Note that a good UX can make this configuration all happen in one-click.
  2. In your proposal, a bot is still needed to convert the structured report into something human-readable.
  3. In your proposal, what's the scenario/UX for a large Community Room, e.g. MatrixHQ? Since these large public rooms are the ones that attract most abuse and spam, I'm hoping to target them pretty quickly for experimentation, with smaller rooms coming later.
  • Leaves a lot of room for flexibility. Maybe instead of forwarding to the "mod room" it files a ticket in a ticketing system.

Well, that's possible with either variant. The main difference in this specific scenario is that the Moderation Room allows more than one bot to operate.

More generally, I believe that the true difference between your proposal and mine is that in yours, the abuse endpoint is a user (which may optionally connect to a Moderation Room, etc.) while in mine, the abuse endpoint is a room (which may optionally connect to users, etc.)

TL;DR I don't see the benefit of mandating the bot and its behaviour.

I believe that I understand your point and that I understand the point of minimizing. If I were to go this way, though, I'd probably come back with a MSC for the bot pretty soon :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does the reporter send the abuse report in your counter-proposal? To a userID as specified in moderated_by?

Yes. You could have a moderated_by which is a list of users. (or a bot)

If I understand correctly, you're splitting the MSC in two.

Basically. I see a lot of value to specifying how the client reports abuse, but I'm not so convinced that the "report management" workflow that you have proposed is sufficient for all use cases, furthermore I don't see as much value in standardizing it. So I think it makes sense to get the reporting flow through. Then we can consider management workflows later if we find value in standardizing it.

Note that a good UX can make this configuration all happen in one-click.

But who is expected to run the bot? Is it to be built into every homeserver?

In your proposal, a bot is still needed to convert the structured report into something human-readable.

Why can't this just be done by the client for the non-bot case?

In your proposal, what's the scenario/UX for a large Community Room, e.g. MatrixHQ? Since these large public rooms are the ones that attract most abuse and spam, I'm hoping to target them pretty quickly for experimentation, with smaller rooms coming later.

In this case we would probably want a bot because with a lot of reports new rooms for each new report may be undesirable. So set moderated_by to the bot address. I don't really know how the mod teams of MatrixHQ work so I don't know if they would want it sent to a mod room like you have described, or they would prefer other options such and filing tickets or sending email.

My point here is that the bot workflow works even without being part of the spec. Each mod team can use a bot that works for them instead of mandating a single bot that implements one workflow.

If I were to go this way, though, I'd probably come back with a MSC for the bot pretty soon

That sounds fine to me. If you can justify the value of specifying the bot I am all ears :)

Copy link
Author

@Yoric Yoric Jun 11, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a room is the abstraction where several tools (including Matrix clients) can connect, not because it's a place to discuss

I get that. But I think it would be nice if it could also be a place to discuss. Basically allow the mods and users to connect directly and discuss if desired, rather than forcing an intermediary. Again, the intermediary is still allowed, but not required.

This also raises your question about responding to the user. Does the mod team reach out directly? Do we specify a method for the bot to reply? The nice thing about not specifying the bot at all is that it lets different options be tested, multiple people can make bots that work the way they want. If we specify the bot we need to answer all of these questions upfront.

I believe that neither proposal solves that scenario, because of the difficulty of trusting DMs. I believe that this deserves its own MSC.

I also believe that both proposal allow for such experimentation (especially since each proposal can easily be extended into the other proposal :) ).

Let me try to clarify what I understand about your proposal and my proposal.

Yours:

1. Mod Room - This is where the mods live, the users know the ID of this room but can't necessarily join it.

2. Community Room - This is where the users live (and likely the mods too).

3. Reporting Room - This is created to send a report. It contains the reporter and a bot.

Mine:

1 is not specified. It may exist (like when using a forwarding, spam checking, enhancing bot), but the spec doesn't require it.

3 contains the reporter and the list of "targets" which may be one bot, or may be moderator(s) or may be both. This is the key difference in our proposals, I prefer not to specify exactly where reports go and how they are handled to allow various approaches to be evaluated and see which approaches work for which mod teams.

Agreed on the summary.

My point here is that hardcoding the appearance in the client is actually less flexible than letting a bot decide of the appearance.

The big difference is how many bots can be used on one mod team. IIUC you can only have one, so the whole team is locked into a style of formatting. If it is done by the client each mod can pick their preferred client. Additional for "small" communities the mods probably wont be bothering to pick their favourite bot so they will just get whatever comes with their homeserver or integration manager (or wherever the bot comes from, it isn't really specified here).

Good point.

Unfortunately, I believe that we have reached a stage at which this conversation has stopped progressing. We both have arguments that make sense, we each appreciate the other's arguments, but I feel that continuing this thread will simply block everything.

So, to summarize:

  1. I believe that we agree that both proposals make sense, that both support experimentation and that each proposal may later be amended to essentially become/encompass the other.
  2. Specifically, it feels like your proposal may be better suited to small Community Rooms but may need to encompass my proposal to be better suited to large Community Rooms. Conversely, it feels like my proposal may be better suited for large Community Rooms and may need to encompass my proposal to be better suited to small Community Rooms.
  3. My first priority is to help deal with the abuse that crops up in large Community Rooms. This is a clear, present and pressing problem.

For these reasons, I believe that continuing work on this MSC more or less as is (i.e. my proposal) does not harm your proposal and should yield clear benefits in terms of both enabling experimentation (including experimentation on your proposal) and aiding the fight against abuse.

Therefore, I'm planning to:

  • stop the current conversation (although we may very well continue it on another channel);
  • proceed by experimenting on this MSC;
  • if experimentation proves successful, remove the "draft" tag and try to move this MSC towards standardization;
  • all of this while keeping the door open for a further MSC based on your proposal.

Does this make sense?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the point that you are missing is that I think specing out the "second half" right now of this MSC is (mildly) harmful. I agree that what I am proposing is a subset of yours. But I think that it makes sense to start with that subset.

Most importantly I am still do not see the benefits that you see in specifying the "second half". If there is no benefit of nailing something down in the spec then I think it is best not to specify it to avoid unnecessary restrictions that may come back to bite us down the road.

Would it help if I put forward a stripped down version of this and we can consider deferring this for now?

each proposal may later be amended to essentially become/encompass the other

It isn't clear that your proposal can be cleanly cut down to the minimal version. Could you clarify roughly the tweaks that you would make? It is just changing the bot name to a MXID or list of MXID?

My first priority is to help deal with the abuse that crops up in large Community Rooms. This is a clear, present and pressing problem.

Yes, that is why I am suggesting pushing out the "first half". Then you can write the bot and use it for these communities. This looks like the fastest path to me and avoids adding technical debt.

So to be clear this is the plan that I think makes the most sense:

  • proceed by pushing the "reporting" half of this MSC.
  • Experiment with the bot, solving the problem for these communities.
  • If we find reasons why we need to specify the "second half" of this MSC then propose those in another MSC.

Again, if you can specify clear reasons why the MSC would be worse without specifying the behaviour of the bot then I think it can go ahead. But reading back I still don't see any reasons why the bot needs to be specified. I have only see "big communities will need it". That statement may well be true but that doesn't mean that it needs to be specified. My preferred subset allows using a bot, and it may well be the case that all users would use a bot, but unless there is a downside to removing the bot from the proposal then I think we should. Simpler is better.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with reducing the MSC to its first half and keeping the second half as an illustration of a possible workflow.

Before experimentation, I'm not ok with specifying that the client must be able to display abuse reports or that we can specify several targets in moderated_by.

Does this work for you?

Copy link
Author

@Yoric Yoric Jun 18, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few additions:

  • I'm currently in the process of experimenting with the current MSC;
  • I believe that to make moderator_of and moderated_by most generic for a MSC v1, their content probably shouldn't be specified. Rather, it is an agreement between the Community Room, the Moderation Room and the bot and/or client.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good.

I'm not ok with specifying that the client must be able to display abuse reports

Then we can consider things like a fallback with extensible events in the future or mandating the bot. For now we can assume that anyone who ads the metadata has a way to read the reports.

that we can specify several targets in moderated_by

Can we cut down the middle for now and say that it must be a list of one element? That way we don't need to break the API to allow multiple recipients. What is your objection to allowing multiple? It doesn't seem to cause any issues in my mind.

I believe that to make moderator_of and moderated_by most generic for a MSC v1, their content probably shouldn't be specified. Rather, it is an agreement between the Community Room, the Moderation Room and the bot and/or client.

That sounds good. We can also consider adding back moderated_by as a more opaque attribute if we want that can be a room ID for a bot or whatever else is desired. But for now I think it is fine to prototype without it and see what is actually desired there. Simple is good :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having a single bot for routing moderation reports is a clear single point of failure, as if the server running the bot goes down, no moderator from any server will be able to receive reports.

}
// ... usual fields
}
```

### Client behavior
- Each room MAY have state events `m.room.moderator_of`. A room that has a state event `m.room.moderation.

```jsonc
Yoric marked this conversation as resolved.
Show resolved Hide resolved
{
"state_key": "m.room.moderation.moderator_of.XXX", // XXX is the ID of the Community Room, i.e. the room being moderated.
Yoric marked this conversation as resolved.
Show resolved Hide resolved
"type": "m.room.moderation.moderator_of",
"content": {
"user_id": XXX, // The bot in charge of forwarding reports to this room.
}
// ... usual fields
}
```

### Client behavior

#### Opting in for moderation

When a user Alice creates a room or when a room moderator accesses the room's configuration, they MAY opt-in for moderation.
When they do, they MUST pick a moderation room. The client SHOULD check that the moderation room is a room in which Alice
has a powerlevel sufficient for sending messages.
When a user Alice creates a room ("the Community Room") or when a room moderator accesses the Community Room's configuration,
they MAY opt-in for moderation. When they do, they MUST pick a Moderation Room. The Client SHOULD check that:
- the Moderation Room is a room in which Alice has a powerlevel sufficient for sending messages;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does Alice need to be able to send messages in the moderation room? What if people configure their moderation room to be a read-only stream of reports (disabling users other than the bot user from sending messages).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I should have written "events" instead of "messages".

  1. Alice needs to set (m.room.moderator_of, room_id).
  2. My intuition tells me that this helps us rely on existing PL mechanisms to allow kicking Alice out of the Moderation Room if they misbehave.

Do you think I'm missing something?

- the Moderation Room has a state event `m.room.moderation.moderator_of`.

This room ID is materialized as a state event `m.room.moderation_room`, as described above.
If Alice has opted-in for moderation, mased on the Moderation Room's Room ID and `m.room.moderation.moderator_of`, the Client
Yoric marked this conversation as resolved.
Show resolved Hide resolved
MUST create a state event `m.room.moderated_by` (see above) in the Community Room.

Similarly, if a moderator has opted in for moderation in a room, a moderator MAY opt out of moderation for that room.
This is materialized as deleting `m.room.moderation_room`.
Similarly, if a moderator has opted in for moderation in a Community Room, a moderator MAY opt out of moderation for that
Community Room. This is materialized as deleting `m.room.moderated_by`.

#### Rejecting moderation

A member of a Moderation Room may disconnect the Moderation Room from a Community Room by removing state event
`m.room.moderation.moderator_of.XXX`. This may serve to reconfigure moderation if a Community Room is deleted
or grows sufficiently to require its dedicated moderation team/bots.

#### Reporting an event

Any member of a room that supports moderation MAY report an event from that room, by sending a `m.abuse.report` event
Any member of a Community Room that supports moderation MAY report an event from that room, by sending a `m.abuse.report` event
with content

| field | Description |
|----------|-------------|
| event_id | **Required** id of the event being reported. |
Yoric marked this conversation as resolved.
Show resolved Hide resolved
| room_id | **Required** id of the room in which the event took place. |
| moderated_by_id | **Required** id of the moderation room, as taken from `m.room.moderated_by`. |
| nature | **Required** The nature of the event, see below. |
| comment | Optional. String. A freeform description of the reason for sending this abuse report. |

`nature` is an enum:

- `abuse.disagreement`: disagree with other user;
- `abuse.toxic`: toxic behavior, including insults, unsollicited invites;
- `abuse.illegal`: illegal behavior, including child pornography, death threats,...;
- `abuse.spam`: commercial spam, propaganda, ... whether from a bot or a human user;
- `abuse.room`: report the entire room, e.g. for voluntarily hosting behavior that violates server ToS;
- `abuse.other`: doesn't fit in any category above.
- `m.abuse.disagreement`: disagree with other user;
- `m.abuse.toxic`: toxic behavior, including insults, unsollicited invites;
- `m.abuse.illegal`: illegal behavior, including child pornography, death threats,...;
- `m.abuse.spam`: commercial spam, propaganda, ... whether from a bot or a human user;
- `m.abuse.room`: report the entire room, e.g. for voluntarily hosting behavior that violates server ToS;
- `m.abuse.other`: doesn't fit in any category above.

We expect that this enum will be amended by further MSCs.

The rationale for requiring a `nature` is twofold:

- a Client may give to give a users the opportunity to think a little about whether the behavior they is truly abuse;
- a Client may give to give a users the opportunity to think a little about whether the behavior they report truly is abuse;
Yoric marked this conversation as resolved.
Show resolved Hide resolved
- this gives the Client the ability to split between
- `abuse.room`, which should be routed to an administrator;
- `abuse.room`, which should be routed to an administrator (in the current MSC, using the existing moderation API);
- `abuse.disagreement`, which may better be handled by blurring messages from offending user;
- everything else, which needs to be handled by a room moderator or a bot.

Any `m.abuse.report` message sent to a moderation room is an abuse report.

This proposal does not specify behavior when `m.room.moderation_room` is not set or when the room doesn't exist.
To send an `m.abuse.report`, the Client posts the `m.abuse.report` message as DM to the `user_id` specified in the
`m.room.moderated_by`.

This proposal does not specify behavior when `m.room.moderated_by` is not set or when the `user_id` doesn't exist.

### Server behavior
### Built-in routing bot behavior

#### Routing messages
Users should not need to join the moderation room to be able to send `m.abuse.report` messages to it, as it would
let them snoop on reports from other users. Rather, we introduce a built-in bot as part of this specification: the
Routing Bot. This Routing Bot is part of the server and has access to priviledged information such as room membership.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, i think this is a bad idea when the focus of matrix wants to shift away from servers and onto users and rooms, maybe specify that such a bot can either be server-controlled or "self-hosted", and it'll solve a lot of problems down the line (such as the "permissions bot" in #2962 which both needs to be verifiable in a decentralized manner, but also built-into every server it'll touch)

State that the bot can be any valid matrix user, then it only has to follow below behaviour to be "acceptable" as a cog in this MSC.

Note that this is a point of centralisation, though i think this is less of a problem than requiring built-in server bots

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow what you're suggesting.

I've removed the "privileged" part, though.


When user Alice attempts to send a `m.abuse.report` message _M_ to room _R_:
1. When the Routing Bot is invited to a room, it always accepts invites.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I'm writing this regardless of the status of the MSC in case it gets picked up again later by someone else, even if that's in another form.)

It would be really useful for the client to give the room a distinct type. Currently in Mjolnir (which has a partial implementation of the routing bot) this behaviour is problematic as it clashes with the acceptInvitesFromSpace behaviour and also protectAllJoinedRooms. matrix-org/mjolnir#475

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't be happy with a solution that requires a bot on my homeserver joining all rooms it's invited into. This seems too abusable. I want my server only participating in rooms that my users explicitly joined.

2. When the Routing Bot receives a message other than `m.abuse.report`, it ignores the message.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be clear, this is an event with type m.abuse.report, rather than an m.room.message event with "msgtype": "m.abuse.report", correct?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, it's a message event with type m.abuse.report.

3. When the Routing Bot receives a message _M_ with type `m.abuse.report` from Alice:
- If the Routing Bot is not a member of _M_`.moderated_by_id`, reject the message.
- If Alice is not a member of _M_.`room_id`, reject the message.
- If room _M_.`moderated_by_id` does not contain a state event `m.room.moderation.moderator_of.XXX`, where `XXX`
is _M_.`room_id`
- Reject the message.
- Otherwise
- Call _S_ the above state event
- If _S_ does not have type `m.room.moderation.moderator_of`, reject the message.
- If _S_ is missing field `user_id`, reject the message.
- If _S_.`user_id` is not the id of the Routing Bot, reject the message.
- If event _M_.`event_id` did not take place in room _M_.`room_id`, reject the message.
- If Alice could not witness event _M_.`event_id`, reject the message.
- Copy the message to room _M_.

- if Alice is not a member of _M_`.room_id`, reject the message;
- if room _M_.`room_id` does not have a state event `m.room.moderation_room`, reject the message;
- if room _M_.`room_id` has a state event `m.room.moderation_room` and its value is other than _R_, reject the message;
- if event _M_.`event_id` did not take place in room _M_`.room_id`, reject the message;
- if Alice could not witness event _M_.`event_id`, reject the message;
- otherwise, send the message to room _R_ **even if Alice is not a member of room _R_**.

**Note** This may needs a new API comparable to https://spec.matrix.org/unstable/server-server-api/#knocking-upon-a-room . To be specified.

### Possible bot behavior
### Possible Moderation Bot behavior

This section is provided as an illustration of the spec, not as part of the spec.

A possible setup would involve two bots, both members of a moderation room _MR_.
A possible setup would involve two Moderation Bots, both members of a moderation room _MR_.
Yoric marked this conversation as resolved.
Show resolved Hide resolved

- A classifier bot consumes `m.abuse.report` messages, discards messages from users who have joined recently or never
- A Classifier Bot consumes `m.abuse.report` messages, discards messages from users who have joined recently or never
been active in the room (possible bots/sleeping bots), then collates reports against users. If there are more than
e.g. 10 reports in the last hour against a single user, post a `m.policy.rule.user` message in the same room specifying that the user
should undergo temporary ban.
- Another bot consumes `m.policy.rule.user` messages and implement bans.

## Open questions

- If all the moderators of room _R_ leave its moderation room _MR_ or are kick/banned from _MR_, we can end up with an orphan
room _R_, which sends its moderation on _MR_ but doesn't have moderators in _MR_. Do we need to handle this?
- Should we allow the members or moderators of a moderation room _MR_ to reject a room _R_ from moderation? If so,
how do we implement this?
- A Ban Bot consumes `m.policy.rule.user` messages and implements bans.

## Security considerations

Expand All @@ -176,7 +203,8 @@ room. There is the possibility that this mechanism could be abused.
We believe that it cannot readily be abused for spam, as these are structured data messages, which are usually not visible to members
of the moderation room.

However, it is possible that it can become a vector for attacks if combined with a bot that treats said structured data messages.
However, it is possible that it can become a vector for attacks if combined with a bot that treats said structured data messages,
e.g. a Classifier Bot and/or a Ban Bot.

### Revealing oneself

Expand Down Expand Up @@ -209,18 +237,28 @@ As bots are invited to moderation rooms, a compromised bot has access to all mod

## Alternatives

### MSC 2938
MSC 2938 (by the same author) has previously been posted to specify a mechanism for reporting events to room moderators. The current MSC is considered
- simpler to implement;
- more reliable (it does not need to roll out its own federation communication);
- less specialized.
- less specialized/more general.

I am not aware of other proposals that cover the same needs.

### Alternatives to the Routing Bot

The "knocking" protocol is an example of an API that lets users inject state events in a room in which they do
not belong. It is possible that we could follow the example of this protocol and implement a similar "abuse" API.

However, this would require implementing yet another new communication protocol based on PDUs/EDUs, including a
(small) custom encryption/certificate layer and another retry mechanism. The author believes that this would entail
a higher risk and result in code that is harder to test and trust.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for including this - I see now that the weight that comes with this solution is not just a new API, but all the scaffolding that goes along with it (including message secrecy).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

including message secrecy

From what I can tell this is not currently part of the MSC, but it could be in the future.

On the point against adding another handshake, I am not too sure as to what the problem is, as a certificate is not only not part of the MSC as it currently stands, but arguably isn't that useful, since any bad actor could spam reports for events they can see instead, which doesn't make a big difference to the moderators dealing with it.

Not too sure what the trust and testing part is about though, since there is already a precedent for using handshakes to send state to rooms a server is not a part of, so the code to perform it would likely be not much of a new thing, especially when compared to creating the concept of a routing bot.


## Unstable prefix

During experimentation

- `m.room.moderation_room` will be prefixed `org.matrix.msc3215.room.moderation_room`;
- `m.room.moderated_by` will be prefixed `org.matrix.msc3215.room.moderated_by`;
- `m.room.moderator_of` will be prefixed `org.matrix.msc3215.room.moderator_of`;
- `m.abuse.report` will be prefixed `org.matrix.msc3215.abuse.report`;
- `abuse.*` will be prefixed `org.matrix.msc3215.abuse.nature.*`.
Yoric marked this conversation as resolved.
Show resolved Hide resolved