MSC4133: Extending User Profile API with Key:Value Pairs #4133
Conversation
tcpipuk
changed the title
turt2live
added
proposal
This comment was marked as resolved.
This comment was marked as resolved.
|
Looks great, thanks! |
|
Seems we discuss here a key-value CRDT, as keys would need to be updated over time. Personally, I'm all in favor. Matrix Event Graph is isomorphic to Merkle-CRDT that powers OrbitDB (key-value DB is one of the supported CRDT). |
|
@mscbot concern no client implementation for user-defined fields |
There was a problem hiding this comment.
I've got a few more concerns off the back of the latest round of changes. I'd like to give personal thanks to @tcpipuk at this stage for preserving on one of the more difficult MSCs. You're doing a great job of wrangling this.
On the MSC, I am slightly worried it is starting to include an absolute behemoth of implementation details. Some of which feel more like they are rephrasing other parts of the MSC in different language.
On the need for an implementation for user defined fields, I agree. Albeit, I am starting to feel inclined to suggest reserving the namespace in this MSC and creating a successor MSC that deals with the implications (that means leaving u*. effectively ignored until clients choose to support that specific MSC). That would leave clients able to use custom profiles, for defined fields they choose to support.
Anyway, that's my 2c. I suspect the SCT will have their own beliefs on what's right here. Again, good job!
proposals/4133-extended-profiles.md
Outdated
| - Servers are *allowed* to suppress some or all local/remote profile fields they do not wish to | ||
| share with their local users (e.g. if there are moderation concerns during a go-live phase); | ||
| however, server admins *must* disclose to users if they are publishing profile fields on behalf | ||
| of a user over federation that they cannot see, as this could be considered a breach of trust. | ||
| This feature could cause confusion if some users can see fields that other users cannot, so this | ||
| should be used sparingly. |
There was a problem hiding this comment.
In what situation would this occur? This seems extremely unusual. I'd almost say this seems like a bad practice thing to do, and contorts how profiles work.
There was a problem hiding this comment.
I believe Jim raised this as an option for large servers to roll out profile fields, i.e. they'd update to a server that supports them but want to block fields being seen/updated until they were ready.
I agree it feels like a bit of a hack, I'm just not sure whether there's a better way to soft-enable/disable profile fields, or if we should even do that at all?
There was a problem hiding this comment.
Hmm, I'd like to understand what @jimmackenzie had in mind here. This feels like it raises the complexity of profiles AND reduces trust in profiles being accurate to me.
There was a problem hiding this comment.
In the meantime, I've rewritten this to be a lot less "servers can suppress whatever they like, whenever they like" and more "this entire feature can be temporarily disabled, but only sparingly": tcpipuk@4afb8b8#diff-e408d6c936a5ae3f7dd4386ed643830fb2191b459b35cbec73b2987d73cae439R294-R296
There was a problem hiding this comment.
overall, I support the feature this MSC is aiming to bring forward. I've left some more detailed comments in my review, but the summary is:
- There's a few places where the text leaves me a bit lost, or has a very blunt transition. Context from past discussions appears missing or overly included as well. A top to bottom read and edit is recommended to unify the feedback received.
- User-specified namespacing should be moved out to a dedicated MSC, or at least deferred to a future MSC. It opens up way too many doors for consideration, and I don't think that blocking this MSC on those discussions is worthwhile.
- Some words about traffic volume would be good.
Thanks for putting this proposal together, and for handling the ongoing feedback. Looking forward to this being merged, and profiles being fixed (finally)!
proposals/4133-extended-profiles.md
Outdated
|
|
||
| ## Client-Server API Changes | ||
|
|
||
| - **Get a Profile Field** |
There was a problem hiding this comment.
blocking: it would be good to mention that this is modeled off the existing endpoints, and link to those endpoints.
There was a problem hiding this comment.
I have done this up top: 9b4741e#diff-e408d6c936a5ae3f7dd4386ed643830fb2191b459b35cbec73b2987d73cae439R11
Or would you prefer a specific mention in the actual CS API section itself?
proposals/4133-extended-profiles.md
Outdated
| } | ||
| ``` | ||
|
|
||
| *Note**: Setting a `null` value does not delete the key; the key remains with a `null` value. |
There was a problem hiding this comment.
blocking: words to explain this behaviour would be appreciated.
There was a problem hiding this comment.
Is this roughly what you were looking for? 9b4741e#diff-e408d6c936a5ae3f7dd4386ed643830fb2191b459b35cbec73b2987d73cae439L61-R66
proposals/4133-extended-profiles.md
Outdated
| if permitted by the homeserver. | ||
|
|
||
| - **Get All Profile Fields** | ||
| - **Endpoint**: `GET /_matrix/client/v3/profile/{userId}` |
There was a problem hiding this comment.
I think this endpoint already exists? Linking to it would be best.
There was a problem hiding this comment.
Great shout, I've done this here: 9b4741e#diff-e408d6c936a5ae3f7dd4386ed643830fb2191b459b35cbec73b2987d73cae439R77-R78
proposals/4133-extended-profiles.md
Outdated
| There is no method to verify the history of global profile fields over federation, so this endpoint | ||
| MUST only accept requests for local users on the current homeserver, and homeservers MUST only | ||
| request a profile from the homeserver specified in that user's MXID. |
There was a problem hiding this comment.
I'd suggest moving the verification bit to an explainer in Potential Issues, and dropping everything after "so this endpoint..." because it's just repeating the spec itself.
There was a problem hiding this comment.
I have done this here: 9b4741e#diff-e408d6c936a5ae3f7dd4386ed643830fb2191b459b35cbec73b2987d73cae439R291
proposals/4133-extended-profiles.md
Outdated
| namespaces. | ||
|
|
||
| - Clients SHOULD only display profiles of users in the current room whose membership status is | ||
| `join`, `invite`, or `knock`, and whose `m.room.member` event has *not* been redacted. If a |
There was a problem hiding this comment.
(clients may not be able to determine if the membership event has been redacted - just excluding leaves, kicks, and bans is probably fine)
There was a problem hiding this comment.
I have adjusted this here: 9b4741e#diff-e408d6c936a5ae3f7dd4386ed643830fb2191b459b35cbec73b2987d73cae439L279-R283
proposals/4133-extended-profiles.md
Outdated
| completely new API specifically for extended information. However, these approaches could lead | ||
| to more significant changes to the existing API and higher complexity for client developers. | ||
|
|
||
| ## Security Considerations |
There was a problem hiding this comment.
on the thread overall: moderation/safety tooling will likely have to adapt to scan-on-read behaviour rather than scan-constantly or scan-on-join. The server is likely best positioned to have this tooling, and can correlate API calls for server-side moderation purposes.
For example:
- Client requests profile of a user.
- Before responding, the server scans the profile and omits obvious violations (possibly auto-reporting internally too).
- Client receives amended profile, displays to user.
- User spots scam link, reports user using relevant API.
- Server sees report was within ~5 minutes of it serving the profile, attaches the profile as-served to the internal ticket for review.
- Server moderator sees both as-served profile and the related ticket for field omission, and can make decisions from there.
None of this exists today, but it would be an immensely valuable tool considering this MSC's feature design. This is not something the MSC needs to describe or spell out, but it should reference that moderation and safety tooling will need to consider the caching semantics of profiles in their design.
There was a problem hiding this comment.
This MSC appears to have gone through a few rounds of comments, and organically iterated upon as a result. It's a bit difficult to read from the dozens of small edits, but I think I get the idea the proposal is trying to put forward.
It's not necessarily a blocker for FCP, but I would encourage a top to bottom read/edit to ensure the paragraphs and sections flow well together and create a cohesive idea. I've annotated a couple of places which need particular attention in my review for reference.
There was a problem hiding this comment.
Thanks for this. I've adopted the specific edits mentioned, and also reorganised sections and corrected some unusual formatting/phrasing in the hope that it helps make the overall document a bit easier.
|
(please avoid force pushing as it makes review significantly harder) |
| - **When capability is missing**: Clients SHOULD assume extended profiles are supported and that | ||
| they can be created or modified. If a server intends to deny some (or all) changes, it SHOULD use | ||
| the capability to advertise this, improving the client experience. |
There was a problem hiding this comment.
I don't think this is backwards compatible -- if it is missing shouldn't it be assumed to be not supported?
Or maybe it should be assumed to be enabled, but the only allowed fields depend on m.set_displayname and m.set_avatar_url?
There was a problem hiding this comment.
That was primarily off the back of this thread: #4133 (comment)
There was a problem hiding this comment.
Maybe this just needs to state that when the capability is missing, but the homeserver's supported spec versions supports custom profile fields?
| - **`M_TOO_MANY_KEYS`**: Exceeds key limits. | ||
|
|
||
| ```json | ||
| { | ||
| "errcode": "M_TOO_MANY_KEYS", | ||
| "error": "The user has exceeded the maximum number of allowed keys in their profile." | ||
| } | ||
| ``` |
There was a problem hiding this comment.
Same here -- what is the maximum number of keys? If we have the 64 KiB limit why do we have this also?
There was a problem hiding this comment.
I think this was an example, so that a homeserver may optionally limit the number of keys a user may create, so clients know to recognise this specific type of situation when it crops up. Should I add explanation for its existence, or just remove it? 🤔
There was a problem hiding this comment.
I think the size of a key and the total size is probably enough limitations? 🤷
| - **Description**: Merge the provided JSON object into the user's profile, updating any | ||
| specified keys without altering others, if permitted by the homeserver. |
There was a problem hiding this comment.
To clarify -- is this a full merge (e.g. RFC 7396) or replacing any values by the top-level key?
For example, if the current values is:
{
"initial": "untouched",
"foo": {"bar": "test"}
}And a PATCH request is received:
{
"key": true,
"foo": {"baz": 1}
}What's the resulting JSON?
Full merge
{
"initial": "untouched",
"key": true,
"foo": {
"bar": "test",
"baz": 1
}
}Top level merge
{
"initial": "untouched",
"key": true,
"foo": {"baz": 1}
}There was a problem hiding this comment.
Good point. I'd assumed it would overwrite the top-level key, but I guess that also invites the same race conditions we've been trying to avoid with the single-key operations.
One worth asking the SCT, perhaps? Is merging trees behaviour we want to protect in Matrix, or should we treat each key as an insular piece of data that should always be overwritten entirely?
There was a problem hiding this comment.
I didn't get much traction from the SCT about which way to go.
@tulir Do you know what hungryserv does? The Synapse PR does just a top-level merge.
I think ways forward are to:
- Just choose one and see what people think.
- Move
PATCHinto a separate MSC.
There was a problem hiding this comment.
Looks like hungry does a top-level merge too
There was a problem hiding this comment.
I'm not sure I see the benefit of PATCH here, whatever merging method we use is going to cause confusion. I think the easiest thing to do is to remove support for this and just have clients either re-upload or set individual keys.
There was a problem hiding this comment.
@tcpipuk To be clear -- it would be best to move PATCH support to a separate MSC to avoid blocking the core of this MSC.
| To minimise the impact of abuse, clients SHOULD offer suitable defaults for displaying user-entered | ||
| content. A user MAY choose to display fields from all users globally, but *by default* profiles | ||
| SHOULD only be shown when the users share the current room and the other user is in the `join`, | ||
| `invite`, or `knock` membership states. |
There was a problem hiding this comment.
I'm not sure about knock here, given for knockable rooms this can be done by any random Matrix user and so there is no extra "trust" gleaned from the fact you "share" a room?
There was a problem hiding this comment.
My thinking was that you'd want to see the fields to help decide if you should allow them to enter.
| - Servers SHOULD cache remote profiles for at least 5 minutes after retrieval. Until a method for | ||
| servers to notify each other of updates is established, it is recommended that profiles be cached | ||
| no longer than an hour to avoid displaying stale data. Servers SHOULD NOT cache profiles for | ||
| longer than 24 hours to avoid unwanted data persistence. |
There was a problem hiding this comment.
I'm not sure this is really needed if this is triggered by a user-driven API requests? Generally we're OK with doing a federation request in that case, as federation requests aren't that expensive. The issue is when servers end up doing automated tasks that end up doings lots of requests.
The downside with recommending caching is that it makes the UX materially worse over federation
There was a problem hiding this comment.
That's good to know. I do think there's some room for optimisation (e.g. if someone joins a room with 10,000 users in it, we don't really want their server asked for timezone information 10,000 times) but we also don't want stale information.
My preference would be to use an EDU to signal when a profile's been updated, so servers/clients can hold onto profile information rather than needing to repeatedly call it. I've avoided including that in this MSC, but this is the rationale behind including caching because it'll likely be encouraged in a future one.
Perhaps I should switch it to a "may" so it's up to servers whether they cache now (and admins can decide how much to compromise UX for performance) but then it's a simple wording change later?
|
@mscbot fcp cancel Per a previous thread, going to cancel on this for now. Once threads are cleaned up and we verify implementations with all the changes we can restart FCP. |
mscbot
added
proposal-in-review
and removed
proposed-final-comment-period
Rendered
Signed-off-by: Tom Foster [email protected]
Known Implementations:
FCP tickyboxes