All bridged IRC users appear as “Offline”

[aperezdc]

Apparently the bridge is missing presence sync, so IRC users appear in Matrix/Riot as “Offline”, even if they are online on the IRC side of the world. This is quite annoying because it causes that people using Matrix/Riot may start talking and mention users in IRC channels without knowing whether the IRC users are online.

Note that this is not about membership syncing, but about presence (online/offline):

• Membership = Making users connected to an IRC channel belong to the bridged Matrix room.
• Presence = When a IRC user connects, mark them as “Online” in Matrix; when they disconnect, mark them as “Offline”.

Both can be independent. For example when not doing incremental membership syncs, having presence syncing would still allow to know who is online and available for talking.

[aperezdc]

While logged in to Freenode the Irssi IRC client as aperezdc__, I joined the channel through the bridge using Riot:

Note how initially there are no other users listed. AFAIU this is okay and intended, and IRC users get added as memnbers of the room when they talk. So I talked from Irssi:

Good, aperezdc__ (IRC) gets added to the room. But notice that the user is marked as “Offline” in Matrix (Riot shows this by painting the user name dim grey in the sidebar), even though I was still connected as aperezdc__ from Irssi.

It would be much better for knowing whether people are around to synchronize people's “presence”, marking them offline when they disconnect from the room, and then back online when they re-connect through IRC to the channel.

[eeeeeta]

I think presence is currently disabled on matrix.org anyway (c.f. comments in #388), so we're probably waiting for it to be turned back on before this could work.

[aperezdc]

@eeeeeta: From the chat in #irc:matrix.org it seems that the bridge only handles membership for now, but noty presence (yet!)

[eeeeeta]

@aperezdc Yes, that seems to be the case - I was just making the point that if it was working, you wouldn't be able to tell, as presence on matrix.org is broken atm (c.f. my message a few messages down in the link you posted)

[kegsay]

So the bridge will already modify presence if the user quits, see https://github.com/matrix-org/matrix-appservice-irc/blob/master/lib/bridge/IrcHandler.js#L640 and https://github.com/matrix-org/matrix-appservice-irc/blob/master/lib/bridge/QuitDebouncer.js#L89 - and the bridge will set the users presence to "online" when it joins - https://github.com/matrix-org/matrix-appservice-irc/blob/master/lib/bridge/IrcHandler.js#L516 - so I'm not really seeing what more can be done here.

Separately, matrix.org has presence disabled currently.

[aperezdc]

@kegsay: I see that the logic for setting users as “Online” can be skipped by this check:

 if (!server.shouldSyncMembershipToMatrix(syncType, chan)) {

This means that presence both presence (online/offline) updates and room membershipo are tied to the very same setting. Wouldn't it make sense to allow updating presence status separately from that setting? We would want to run our instance of the bridge with:

• Presence updates enabled, and
• Room membership updates disabled.

Or would that be nonsensical?

[kegsay]

It's somewhat nonsensical. In an ideal world:

• All bridges would sync membership lists. The reason why people don't is simply because of Synapse problems.
• Some bridges may use presence to avoid flapping members leaving/re-joining constantly which is spammy.

The IRC bridge supports both of those use cases. I cannot fathom a use case where you would want presence to indicate if you were on IRC, and not room membership updates.

[allanday]

This is probably the biggest issue for me with Matrix at the moment. There are times when I've ended up running an IRC client in order to see who is online, so I can chat to them from Matrix.

[ArchangeGabriel]

@kegsay How does your second use case works? I mean, how IRC users get added to the membership list, how do they get removed?

For me the issue is it’s hard to match membership and presence in Matrix with things in IRC. Because in this network there is only one state: present or not.

[kegsay]

How does your second use case works? I mean, how IRC users get added to the membership list, how do they get removed?

• IRC user is connected to a bridged channel. They are in the membership list on Matrix and IRC. They are marked as Online in Matrix.
• IRC user disconnects. Bridge can now either immediately leave the user on Matrix so they are no longer in the membership list, or the bridge can wait to see if they come back. In order to signal to Matrix users immediately that they are no longer there, they are marked as Offline but are still in the membership list.
• After a period of time, if they still haven't returned, they leave the Matrix room.

This works well when connections flap e.g during netsplits, as most users will return.

there is only one state: present or not.

Technically not true, there is the concept of AWAY in IRC.

[aperezdc]

IRC user disconnects. Bridge can now either immediately leave the user on Matrix so they are no longer in the membership list, or the bridge can wait to see if they come back. In order to signal to Matrix users immediately that they are no longer there, they are marked as Offline but are still in the membership list.

@kegsay: How would one go about configuring the bridge to wait a certain time before removing offlined users from the membership list?

AFAIK one can set membershipLists → global → ircToMatrix → incremental: true, but there's no setting to configure the delay before removing IRC users. Am I missing something?

[kegsay]

You are indeed missing something. https://github.com/matrix-org/matrix-appservice-irc/blob/master/config.sample.yaml#L74 - It's sadly hooked pretty deep to netsplit detection though, nor does it seem to detail the presence setting which is being done in the mean time. See:
https://github.com/matrix-org/matrix-appservice-irc/blob/master/lib/bridge/QuitDebouncer.js#L85

[mcatanzaro]

So the use-case for the original request is that (a) we can't currently enable membership sync for fear of overwhelming our Matrix server, and (b) the IRC bridge is currently borderline unusable without it. Being able to see who is currently online is a core requirement for an IRC client.

Now, this seems like a solvable problem because (1) presence updates are unproblematic, and (2) we only need room membership updates when a user joins the room, not when the user leaves. We do not need a membership update when the user leaves the room if the user's presence is updated. That is, it's totally fine for Riot to show a bunch of offline users in the room membership list, so long as they're shown as offline. We still always need membership updates when someone joins a room, but we could make these very rare if we rarely ever remove users' membership. For instance, if we remove inactive users' membership only once per week or month, then it should be relatively rare that new users are added to the room.

[kegsay]

we can't currently enable membership sync for fear of overwhelming our Matrix server

Unless you're bridging all of Freenode, you should be fine. We have membership sync fully enabled on all other IRC networks without issue. The problem comes when you surge join/leave requests (e.g on startup when first enabling it, and on netsplits). "Business as usual" numbers are incredibly low, no higher than 0.1Hz on OFTC for example (1 join/leave every 10 seconds).

The bridge is fairly intelligent when it comes to trickling requests through to avoid surging Synapse. There's the aforementioned quit debouncer https://github.com/matrix-org/matrix-appservice-irc/blob/master/config.sample.yaml#L70 for netsplits, and https://github.com/matrix-org/matrix-appservice-irc/blob/master/lib/bridge/MemberListSyncer.js#L43 is hard-coded for that initial "I've just turned things on and am worried it'll blow up Synapse when it sends a bazillion leave requests".

presence updates are unproblematic

This isn't really true, presence has its own set of problems which led to us disabling it on matrix.org for a while.

[kegsay]

It's worth noting that even with membership list syncing enabled fully, it falls at the last hurdle as it doesn't correctly track nick changes, so you can accrue stale users which will be removed on bridge restarts only (assuming initial leaves are turned on) - see #71

[aperezdc]

@kegsay: Thanks for all the comments. We do have Freenode bridged as well in our instance, so there's that... I'll be trying to reconfigure it with a kind-of-high value for the quit debouncer delay, and see what happens.

But yeah, Freenode 😓

(FWIW, I still see value in propagating presence without enabling membership syncing.)

[mcatanzaro]

@aperezdc surely we only need to sync membership in the rooms that our Matrix users have actually joined...?

[aperezdc]

@kegsay I have tried enabling membership syncing, using a long delay for the quit debouncer, but unfortunately all IRC bridged users still appear as offline. The relevant bits in my config.yaml look like this:

ircService:
  servers:
    irc.gimp.ca:
      # ...
      quitDebounce:
        enabled: true
        delayMinMs: 259200000 # 3 days
        delayMaxMs: 345600000 # 4 days
      # ...
      membershipLists:
        enabled: true
        floodDelayMs: 10000
        global:
          ircToMatrix:
            initial: true
            incremental: true
          matrixToIrc:
            initial: true
            incremental: true

...and I don't see anything relevant in the logs. If I am understanding correctly, the TL;DR is that one has to enable membership syncing, and the “mark offline first and wait $TIME before removing the membership“ behavior should work when the quit debouncer is enabled, using delays configured for it. Did I understand correctly, or is there something else I need to configure?

[aperezdc]

So today I tried killing the bridge, and presence syncing worked. Kinda.

Right after starting back the bridge, presence updates were done for a little while, and after the bridge ran for ~10 minutes suddenly every bridged IRC user was marked as Offline. Could it be that there's some bug related to presence syncing there?

[kegsay]

Quite possible, given we can't check the code is functioning correctly due to presence syncing on matrix.org being down. If you can make a repro case that'd be great. Thanks.

[aperezdc]

@kegsay What would be useful to help debug the presence syncing? Logs from the bridge itself? If so, is there any particular configuration you would like me to apply locally (e.g. enabling debug output)?

[aperezdc]

Also, I keep seeing value in having the possibility of enabling presence updates even if membership syncing is disabled. It would be handy for us, and also for the public bridge, where some networks (like Freenode) do not have membership syncing, and therefore presence updates are disabled as well.

[kegsay]

@aperezdc : yes, logs would be helpful. It sounds like a timeout is expiring and then marking users as offline. The logs will show all outbound HTTP requests so I can see if that's the case or not.

[kegsay]

PM them to me: @kegan:matrix.org.

[ArchangeGabriel]

You are indeed missing something. https://github.com/matrix-org/matrix-appservice-irc/blob/master/config.sample.yaml#L74 - It's sadly hooked pretty deep to netsplit detection though, nor does it seem to detail the presence setting which is being done in the mean time.

@kegsay They indeed aren’t even the words “presence” or “offline” in this config section. ;) Also, I’m not sure how it should work. How do I achieve the following result:

1. All people connected to IRC appear as online on Matrix side, same thing for the reverse direction (so this is all syncing settings to true, right ?).
2. If an user leaves on IRC side, it is shown as offline on Matrix side, with the following subcases:
   a. Until a certain amount of time, after which it is then removed from the user list.
   b. Until bridge restart.
   c. Forever.

That being said, I also have all users shown as offline currently. I don’t really understand what does it means under default settings, since this status doesn’t seem to match anything…

[kegsay]

In general, the bridge doesn't adjust the presence value (online/offline) for (IRC) users on Matrix.

It does in one situation and one situation only: quit debouncing. When quit debouncing is enabled, leaves on IRC will not change presence unless a netsplit was detected in which case instead of leaving the room the bridge will toggle presence to "offline" for a period of time before leaving the room. If the user rejoins the channel before the period of time expires, presence is changed to "online" and they will not leave the room. This means we didn't need to leave/re-join the IRC user, we just toggled presence instead.

This makes for a poor substitute over membership list syncing because clients will still be tab-completing the "offline" user and they will still be present in the membership list, which is why it isn't an option separate to quit debouncing. EDIT: In addition, presence changes don't show up on the timeline so it's unclear whether or not someone received certain messages which appear in scrollback.

tl;dr @ArchangeGabriel You cannot currently achieve the result you outline.

[ArchangeGabriel]

@kegsay At least 2.a and either 2.b or 2.c (dunno what is the behaviour here, but I suppose b.) should be achievable by setting quitsPerSecond to 1 or am I wrong?

Because if I understand well IRC users should be marked as Online by default, not Offline which should only happen in case of netsplit indeed.

So if I’m right above, my only issue then is that IRC users are not marked Online.

[strk]

At the moment the bridge does report join/part events, is that not enough to toggle presence?

[heftig]

It would be nice if AWAY on the IRC side was synced with presence on the Matrix side, in both directions.

[bugz8unny69]

This issue hasn't been closed, so I am interested to know what this stands? Is presence still disabled? When I startup the bridge, users who in the room already, when they are speaking, still appear offline ?

[heftig]

I have a presence enabled homeserver and bridge, and all the IRC users show up as "offline since <when they last sent a message>".

[ronilaukkarinen]

It would be nice if AWAY on the IRC side was synced with presence on the Matrix side, in both directions.

Second to this.