Unifiedpush #3448

p1gp1g · 2021-06-02T06:29:13Z

Pull Request Checklist

Changes has been tested on an Android device or Android emulator with API 21
UI change has been tested on both light and dark themes
Pull request is based on the develop branch
Pull request includes a new file under ./newsfragment. See https://github.com/vector-im/element-android/blob/develop/CONTRIBUTING.md#changelog
Pull request includes screenshots or videos if containing UI changes
Pull request includes a sign off

This new PR for UnifiedPush (old: #2993) is rebased, have a better git flow and the previously missing parts are done.

A phone without a Distributor:
- The gplay flavor will use FCM if it is available, else it will use the background sync.
- The fdroid flavor will use the background sync.
A phone using backgroundsync can use push by installing an unifiedpush distributor without relogging/wiping data.
It is possible to go again to backgroundsync if all distributor are uninstall (and it is not gplay flavor with a working FCM, since the fallback is FCM)
Settings and troubleshoot list are updated whether it is [ gplay flavor with valid FCM or fdroid flavor with unifiedpush distributor ] or [ gplay flavor without FCM or fdroid flavor without any distributor ].

p1gp1g · 2021-06-03T23:26:26Z

~~The last commit enable to use sygnal for push with the embedded distrib. It needs this pushkin, which is mainly the gcmpushkin with a little modification on the data field.~~

Note: embedded distrib means FCM if nothing else is installed (or if the user want this), so probably most of users

Edit: Not true since 4a8fa4a

p1gp1g · 2021-06-07T11:26:35Z

The last commit create a different pusher app id whether it uses the embedded distrib or not. It is now possible to have a pushkin for the actual main version (GCM), this PR version using the embedded distrib (FCM), and this PR version with unifiedpush.

p1gp1g · 2021-06-30T17:56:30Z

The last commit allows to use the current gateway as it is (sygnal with gcmpushkin) for the FCM embedded distrib.

bmarty

Thanks for the PR, it looks promising. One first question

vector/src/gplay/java/im/vector/app/push/fcm/NotificationTroubleshootTestManagerFactory.kt

bmarty

Some other comments.
Also can you please review https://buildkite.com/matrix-dot-org/element-android/builds/2686#8c47ad6f-354f-4f9e-b0e0-e87c23230d22?
And rebasing the PR should fix the lint issue about "radio".

vector/src/gplay/java/im/vector/app/push/fcm/NotificationTroubleshootTestManagerFactory.kt

vector/src/fdroid/java/im/vector/app/push/fcm/NotificationTroubleshootTestManagerFactory.kt

vector/src/gplay/java/im/vector/app/push/fcm/NotificationTroubleshootTestManagerFactory.kt

vector/src/main/java/im/vector/app/core/pushers/UPHelper.kt

vector/src/main/java/im/vector/app/core/pushers/VectorMessagingReceiver.kt

vector/src/main/java/im/vector/app/features/settings/troubleshoot/TestNewEndpoint.kt

bmarty · 2021-07-05T19:06:30Z

vector/src/main/res/values/config.xml

@@ -15,6 +15,8 @@

    <!-- Note: pusher_http_url should have path '/_matrix/push/v1/notify' -->
    <string name="pusher_http_url" translatable="false">https://matrix.org/_matrix/push/v1/notify</string>
+    <!-- Note: pusher_http_url should have path '/_matrix/push/v1/notify' -->
+    <string name="default_push_gateway_http_url" translatable="false">https://matrix.gateway.unifiedpush.org/_matrix/push/v1/notify</string>


Update the comment?

One remark, but an important one, what happen if this service is down?

This push gateway is used to forward notifications to users' UnifiedPush endpoints and is not used by the FCM embedded distributor. So if it is down, users using another UnifiedPush distributor than the embedded one (which will be used by most users) won't receive push notifications. Just like users using FCM embedded distributor won't receive push notif if the matrix sygnal server is down.

This push gateway is the unifiedpush common-proxies on our server. It can be hosted by element/matrix team. It is also possible to write a sygnal pushkin for that: it should be quite easy to write (with a big warning on SSRF).

But if I want to use my server, without participation unifiedpush.org?

@manchelsi You can self host a push proxy like gotify or nfty alongside common-proxies (only required for gotify), install the apps, point them to your server, and tell element to use the app as a distributor.

vector/build.gradle

p1gp1g · 2021-07-07T08:24:46Z

It should be fine now :)

bmarty · 2021-07-09T07:50:32Z

vector/build.gradle

+    gplayImplementation('com.github.UnifiedPush:android-embedded_fcm_distributor:1.1.0') {
+        exclude group: 'com.google.firebase', module: 'firebase-core'
+        exclude group: 'com.google.firebase', module: 'firebase-analytics'
+        exclude group: 'com.google.firebase', module: 'firebase-measurement-connector'


It's weird that we have to exclude those modules again, since it's already done here: https://github.com/UnifiedPush/android-embedded_fcm_distributor/blob/main/lib/build.gradle#L44

Anyway, the CI is happy now, thanks

Yes, that's very weird. And when the lib is a module of the project it isn't necessary. That's probably because of jitpack then

bmarty · 2021-07-12T10:26:18Z

Hello @p1gp1g ,

Thanks for the update of the PR.

Can you explain what will happen when the user will upgrade their GPlay application with a new version with this PR included? Does the app will have to register again the pusher? I'd like to ensure that this scenario is currently handled, and that the user will still receive push as normal.

Thanks

p1gp1g · 2021-07-12T14:50:38Z

Can you explain what will happen when the user will upgrade their GPlay application with a new version with this PR included? Does the app will have to register again the pusher? I'd like to ensure that this scenario is currently handled, and that the user will still receive push as normal.

If the user doesn't have any unifiedpush distibutor (eg. gotify-up) it is transparent. I have tested the migration from main branch to this PR.

The reason is every time the application is started, it does the unifiedpush registration (to the saved distributor if it exists). So it receives a new endpoint and register the pusher again (https://github.com/vector-im/element-android/pull/3448/files#diff-89945d3e11caf4a6cddd30ff052082001992c86186660ee2e1c9093f04c87f30R164).
Maybe a check can be done to register only if the key (FCM Key or UnifiedPush endpoint) is different from previously before register (~~PushersManager.registerPusher~~VectorMessagingReceiver).

Edit: check done with dbfa5e1

p1gp1g · 2021-07-12T14:52:14Z

We can mention issue #224 (in the roadmap too) :

https://github.com/vector-im/element-android/pull/3448/files#diff-44923f4e9b5a6e14764ee8b524a9d8afaeb48ac8f6cb8d708b0794965c557692R95

The gplay flavor works like the fdroid one if there is no play services

jwsp1 · 2021-07-18T14:19:15Z

I guess the user has to install a unifiedpush app like gotify-up in order to make it work for element?
If yes, is it planned to show a hint in the notification settings? Or troubleshoot setting?

p1gp1g · 2021-07-20T12:35:16Z

I guess the user has to install a unifiedpush app like gotify-up in order to make it work for element?
If yes, is it planned to show a hint in the notification settings? Or troubleshoot setting?

Phones will always have a way to receive notifications, with and without unifiedpush distribtutor. cf #3448 (comment)

But it is possible to add a check on the troubleshoot list that say : the distributor used if any, no distributor is found if it uses the background sync. I don't know if it is useful since it is already viewable : the troubleshoot list displays the endpoint or background sync information

jwsp1 · 2021-07-28T07:50:39Z

Ah I see, thanks for replying. Hope this gets merged soon - don't get any notifications from Element (F-Droid, Lineage with microG). Looking at PlayStore reviews, I'm not the only one (although oddly, they should have fcm).
Addressing notification issues should be prio 1 for an >instant< messenger.

jwsp1 · 2021-08-15T16:03:45Z

In TWIM, you said that we should tell you if we didn't get a push notification last week... I didn't get one (Element Android, F-Droid).
I think this PR could be a solution.

Also: If you've received any push notifications in Element Android or Element iOS this week, they've come via our shiny new stateless release of Sygnal! If you haven't received expected push notifications on those platforms this week... please alert us immediately. 😉

Source: TWIM 2021-08-13

genofire · 2021-09-07T21:18:10Z

Now there is an merge conflict 😢 - I really wish this PR is merged soon (On F-Droid this is not an instant messaging 😭 , nearly useless for Messaging )

Thank you for the great work till yet @p1gp1g

jacktheripper19 · 2021-09-08T10:15:04Z

Now there is an merge conflict - I really wish this PR is merged soon (On F-Droid this is not an instant messaging , nearly useless for Messaging )

Thank you for the great work till yet @p1gp1g

Yes it's frustrating. This PR has been open for a long time. There was a time when there was no merge conflicts. If you're not patient, there is the beta version of SchildiChat which is a fork of Element. It had UnfiedPush implemented in it.

p1gp1g · 2021-09-08T11:31:47Z

The conflict is just another addition on the build.gradle, nothing relevant 😉

genofire · 2021-09-08T11:33:19Z

Really SchildiChat#68 is still open and say it is an upstream issue? Or do you mean https://fluffychat.im ?

SpiritCroc · 2021-09-08T11:36:27Z

Really SchildiChat#68 is still open and say it is an upstream issue? Or do you mean https://fuffychat.im ?

"Upstream" issues do not prevent SchildiChat from implementing it anyways. It's still open as it's not enabled in stable/production builds yet.

p1gp1g · 2021-09-16T21:09:35Z

Is there a reason why we don't get events in fast lane on push notification with a sync as a fallback ? (cf 94f9f7f) Should I merge this commit here ?

bmarty · 2021-10-04T21:17:04Z

Is there a reason why we don't get events in fast lane on push notification with a sync as a fallback ? (cf p1gp1g@6cd73da) Should I merge this commit here ?

This commit is not OK to me, a sync is often necessary, to be able to decrypt encrypted messages, or to handle other messages, which was not triggering a Push.

p1gp1g · 2021-10-07T07:30:04Z

Is there a reason why we don't get events in fast lane on push notification with a sync as a fallback ? (cf p1gp1g@6cd73da [/p1gp1g@94f9f7f]) Should I merge this commit here ?

This commit is not OK to me, a sync is often necessary, to be able to decrypt encrypted messages, or to handle other messages, which was not triggering a Push.

Indeed, I've observed that, as it is, the fallback to the sync does not always occured and that's the most important thing.

Moreover, it seems it doesn't go faster most of the time.

Fortunately, I didn't commit it here :)

p1gp1g · 2022-04-26T20:40:24Z

PR is rebased.

For your information, we have released a FOSS library to receive notification via google service [1][2]. This library does not use any proprietary lib, and so it follows FDroid rules. You might want to use it and having a single flavor. If you are interested, I can do it in another PR

Also, it would be great to have some information on the internal discussion state.

[1] https://unifiedpush.org/developers/embedded_fcm/
[2] https://github.com/UnifiedPush/android-foss_embedded_fcm_distributor/

no-usernames-left · 2022-04-26T20:49:44Z

we have released a FOSS library to receive notification via google service

Please correct me if I'm wrong, but I think most of the point of UnifiedPush is because it's not Google.

p1gp1g · 2022-04-26T21:05:36Z

we have released a FOSS library to receive notification via google service

Please correct me if I'm wrong, but I think most of the point of UnifiedPush is because it's not Google.

Indeed, but most users and developers will continue to use Google services. With these libraries (foss & non-foss embedded distributor) users can continue to receive notifications with google services if they want to and it is pretty easy to setup for developers. With the FOSS one, it is achieved without having any proprietary code in the app, so we know there is not any telemetry thing running in the app.

I'd like to avoid too much noise on this PR, if you want to, you're welcome to come in our room : https://matrix.to/#/#unifiedpush:matrix.org

genofire · 2022-05-03T10:48:19Z

A half year without review, comment or so ...
Any news, if this will be merged soon @bmarty ?

bmarty · 2022-06-01T11:54:09Z

Hello @p1gp1g, I will merge it on one on my branch to be able to fix the conflict, do some rework and add some documentation.

Sorry, I do not see the sign-off. Can you add a sign off to the PR, so that I can merge it in our repo please?

Signed-off-by: sim <[email protected]>

bmarty · 2022-06-01T12:52:19Z

Merged in my branch, will do some rework on it and create a new PR.

genofire · 2022-06-02T14:10:23Z

It would be really nice, when we get an issue or an Milestone, where we could trace, that is released.

genofire · 2022-06-02T23:44:30Z

Maybe this issue #2743

p1gp1g mentioned this pull request Jun 2, 2021

Use UnifiedPush for push notifications #2743

Closed

SpiritCroc mentioned this pull request Jun 8, 2021

Unified push SchildiChat/SchildiChat-android#68

Closed

p1gp1g mentioned this pull request Jun 8, 2021

Element Android UnifiedPush/wishlist#4

Closed

a22sc mentioned this pull request Jun 19, 2021

OpenPush for notifications #1002

Open

bmarty reviewed Jul 5, 2021

View reviewed changes

vector/src/gplay/java/im/vector/app/push/fcm/NotificationTroubleshootTestManagerFactory.kt Show resolved Hide resolved

bmarty reviewed Jul 5, 2021

View reviewed changes

bmarty reviewed Jul 9, 2021

View reviewed changes

genofire mentioned this pull request Aug 20, 2021

Unified Push Notifications syphon-org/syphon#252

Open

p1gp1g force-pushed the unifiedpush branch from e4bb12d to 94bfe38 Compare September 16, 2021 18:02

SpiritCroc mentioned this pull request Sep 18, 2021

Question about Unified Push SchildiChat/SchildiChat-android#87

Closed

p1gp1g force-pushed the unifiedpush branch from a520098 to 182db4c Compare October 5, 2021 05:53

p1gp1g added 2 commits April 26, 2022 22:17

Lint

61b2dbd

Check custom unifiedpush gateway

97d13e3

p1gp1g force-pushed the unifiedpush branch from c1801b0 to def4dd1 Compare April 26, 2022 20:32

bmarty changed the base branch from develop to bma/unifiedPush June 1, 2022 12:14

p1gp1g force-pushed the unifiedpush branch from def4dd1 to 8b46a22 Compare June 1, 2022 12:24

Update UnifiedPush libs

8b46a22

Signed-off-by: sim <[email protected]>

bmarty deleted the branch element-hq:feature/bma/unifiedPush2 June 1, 2022 12:27

bmarty closed this Jun 1, 2022

bmarty reopened this Jun 1, 2022

bmarty changed the base branch from bma/unifiedPush to feature/bma/unifiedPush June 1, 2022 12:36

bmarty changed the base branch from feature/bma/unifiedPush to try2 June 1, 2022 12:43

bmarty changed the base branch from try2 to feature/bma/unifiedPush2 June 1, 2022 12:45

bmarty merged commit a8d5040 into element-hq:feature/bma/unifiedPush2 Jun 1, 2022

bmarty mentioned this pull request Jun 9, 2022

UnifiedPush #6228

Merged

15 tasks

Zocker1999NET mentioned this pull request Aug 5, 2022

UnifiedPush for push notifications nextcloud/android#8684

Open

Unifiedpush #3448

Unifiedpush #3448

Conversation

p1gp1g commented Jun 2, 2021 • edited Loading

Pull Request Checklist

p1gp1g commented Jun 3, 2021 • edited Loading

p1gp1g commented Jun 7, 2021

p1gp1g commented Jun 30, 2021

bmarty left a comment

Choose a reason for hiding this comment

bmarty left a comment

Choose a reason for hiding this comment

bmarty Jul 5, 2021

Choose a reason for hiding this comment

bmarty Jul 9, 2021

Choose a reason for hiding this comment

p1gp1g Jul 9, 2021

Choose a reason for hiding this comment

manchelsi Jan 20, 2022

Choose a reason for hiding this comment

minecraftchest1 Jun 19, 2022

Choose a reason for hiding this comment

p1gp1g commented Jul 7, 2021

bmarty Jul 9, 2021

Choose a reason for hiding this comment

p1gp1g Jul 9, 2021

Choose a reason for hiding this comment

bmarty commented Jul 12, 2021

p1gp1g commented Jul 12, 2021 • edited Loading

p1gp1g commented Jul 12, 2021

jwsp1 commented Jul 18, 2021

p1gp1g commented Jul 20, 2021

jwsp1 commented Jul 28, 2021

jwsp1 commented Aug 15, 2021

genofire commented Sep 7, 2021 • edited Loading

jacktheripper19 commented Sep 8, 2021

p1gp1g commented Sep 8, 2021

genofire commented Sep 8, 2021 • edited Loading

SpiritCroc commented Sep 8, 2021

p1gp1g commented Sep 16, 2021 • edited Loading

bmarty commented Oct 4, 2021

p1gp1g commented Oct 7, 2021

p1gp1g commented Apr 26, 2022

no-usernames-left commented Apr 26, 2022 • edited Loading

p1gp1g commented Apr 26, 2022 • edited Loading

genofire commented May 3, 2022

bmarty commented Jun 1, 2022

bmarty commented Jun 1, 2022

genofire commented Jun 2, 2022

genofire commented Jun 2, 2022

p1gp1g commented Jun 2, 2021 •

edited

Loading

p1gp1g commented Jun 3, 2021 •

edited

Loading

p1gp1g commented Jul 12, 2021 •

edited

Loading

genofire commented Sep 7, 2021 •

edited

Loading

genofire commented Sep 8, 2021 •

edited

Loading

p1gp1g commented Sep 16, 2021 •

edited

Loading

no-usernames-left commented Apr 26, 2022 •

edited

Loading

p1gp1g commented Apr 26, 2022 •

edited

Loading