Add support for replying via notifications on macOS

[JoniVR]

Hi,

First of all thanks for this great application.

Fixes #298.

Some things:

• If you do consider merging, some testing is required on Windows and Linux as I do not currently have access to these devices. The inline reply should (I think) only work on macOS as that is the only platform that seems to be supported (see documentation) but correct me if I'm wrong.
• I'm not an expert at electron apps, so forgive any things I overlooked, missed or did wrong, and let me know, I'll fix them.
• If you have any further questions about implementation I'd be happy to answer them.

Also, I made this the default, not configurable yet (since I thought it didn't make much difference in the user experience), but if you want me to I can add it as an extra option.

[CvX]

Alright, I'm testing this change, here's what I found:

1. The reply isn't sent when Caprine's window is closed
2. Same if, window is visible but a different conversation is active
3. If the reply contains any emoji it sends only the first emoji (no text before or after it, or any later emojis)
4. Using setTimeout to wait for the interface is bound to break often. I'd suggest to move this:

   caprine/browser.js

   Lines 559 to 566 in 0f00f08

   	// Wait for Messenger to go to correct message and then start typing and sending
   	setTimeout(async () => {
   	await typeReply(data.reply, data.locale);
   	await sendReply();
   	if (previousConversation) {
   	selectConversation(previousConversation);
   	}
   	}, 50);

   to

   caprine/notifications-isolated.js

   Lines 4 to 18 in 0f00f08

   	// Handle events sent from the browser process
   	window.addEventListener('message', ({data: {type, data}}) => {
   	if (type === 'notification-callback') {
   	const {callbackName, id} = data;
   	const notification = notifications.get(id);
   	if (notification && notification[callbackName]) {
   	notification[callbackName]();
   	}
   	if (callbackName === 'onclose') {
   	notifications.delete(id);
   	}
   	}
   	});

   probably as a new type, e.g. notification-reply (just like we use notification-callback in two contexts). I'm not 100% sure, but I would hope that Messenger switches to to the active conversation synchronously in the onclick handler, which means that moving the reply-typing code here would solve the timing issue. Of course then, the switching back to the previous conversation is also getting more complex, since it has to be split between 'notifications-isolated.js' and 'browser.js' (i.e. after calling onclick and typing the reply, it would window.postMessage() back to the browser context where selectConversation can be called).

I realize that number 4 is a large task. Let's say that after 1-3 are fixed, we'll just try it out and see if the current code is reliable enough. Or maybe doing the 4 will be necessary to fix previous issues… 🤷‍♂️😅

[JoniVR]

Hi @CvX

Thanks for looking into it, I'll fix 1-3 and then look into 4 again, I really struggled with nr. 4.

I wanted to use the callback but couldn't get it to work without doing hacky stuff, so I implemented the least hacky solution that would work, but I'll look into it again. I should be able to get it to work because as you said, setTimeout() is not really a reliable solution.

edit:
Also, for getting the locale, I'm currently reading it from the cookies, but I haven't handled the case where the cookie is not present, how should I go about doing that? I figured I'd just call setUserLocale() inside index.js and then use the following lines again (to prevent a recursive loop):

const facebookLocales = require('facebook-locales');
const userLocale = facebookLocales.bestFacebookLocaleFor(app.getLocale().replace('-', '_'));

or should I move the above two lines to getUserLocale() and then call that function inside setUserLocale()? That actually seems like a better solution I think?

[JoniVR]

1. The reply isn't sent when Caprine's window is closed
2. Same if, window is visible but a different conversation is active

I just tested these again and they seem to work on my end, with "the window closed", you mean when you press the (x) button to just close the window right? Does the text field contain the message you intended to send?

[CvX]

Hm, can't reproduce it now either. Maybe because of the emoji issue I mistook sent message for an earlier one…

[JoniVR]

Okay so the emoji thing is a though one.
I originally tried to implement it using:

document.querySelector('div[contenteditable="true"]').focus();
document.execCommand('insertText', false, 'message 😂😂');

but then it would display:

Happens only on an empty textfield, if there is content already in the textfield it successfully replaces it.

So I used:

const event = document.createEvent('TextEvent');
event.initTextEvent('textInput', true, true, window, message, 0, locale);

instead. However, this is causing issues with the emoji being pasted (and this function is deprecated, so shouldn't be used anyways, I figured that out yesterday).
I originally thought the "Could not display composer." message had something to do with the locale, and so I switched to the other method which allowed me to set a locale, but it doesn't seem like that's the issue.

I've seen the error message on the internet on numerous forums, but it's most of the times related to messenger itself or an extension. I has also been mentioned in #316. Does anyone else have an idea? I'll keep looking into it.

edit:
You can test it out for yourself by pasting the first codeblock into your console in dev tools. If you want to get rid of the message, switch to a different convo and then back, the text will be in the textfield.

edit 2:
It's apparently using Draft.js (which makes sense since facebook developed it), however, Draft.js is written in react, so inserting text is a bit of a problem, that's probably what's causing the "could not display composer message", you can see the actual error by trying it on Draftjs.org.

[CvX]

I have a solution. It's a bit of a hack, but after a couple of hours of research I couldn't find anything better.

What I did is a combination of both your attempts. 😃

const inputField = document.querySelector('[contenteditable="true"]');

if (inputField) {
  inputField.focus();

  const event = document.createEvent('TextEvent');
  event.initTextEvent('textInput', true, true, window, 'placeholder', 0, locale);
  inputField.dispatchEvent(event);

  document.execCommand('selectAll', false, null);
  document.execCommand('insertText', false, message);
}

The clues were all there! 😄TextEvent works for non-emoji text, insertText command works when there's already some text typed in.

Btw, while working on a solution, I thought of an edge case that should be handled. If you have an unsent message typed in, the app should save that message, do the reply logic, and then restore that message.

[JoniVR]

Thanks for looking into it! That would certainly work and is a great workaround!

Another solution to this that I thought of would be to quickly switch between your previous convo and back, because then the error goes away and the inputted text is still present and ready to be sent, this solution is equally hacky but doesn't make use of the deprecated API (and I haven't tested how reliable it is yet).

I was still looking for another way but so far didn't have any luck (even posted a stackoverflow question).

If you have an unsent message typed in, the app should save that message, do the reply logic, and then restore that message.

I agree, that's a good idea, if it's possible to implement it 😅

@sindresorhus Any thoughts on this issue? Should we just implement a somewhat hacky solution?

[CvX]

The conversation-switcheroo is a nice solution. It is certainly much cleaner and more straightforward than using deprecated events and a couple of text-manipulating commands. There are two downsides that come to mind, though:

1. what if you have just a single conversation? 😃
2. a scenario where Caprine's window is present (but not in focus) – the conversation switch would be visible

Restoring the previous message is possible, just did a quick test. The bottom line is: document.querySelector('div[contenteditable="true"]').textContent is just enough.
Here's the full example:

const messageInput = document.querySelector('div[contenteditable="true"]');
let previousMessage = messageInput.textContent;

messageInput.focus();

if (previousMessage) {
  document.execCommand('selectAll', false, null);
  document.execCommand('delete', false, null);
}

document.execCommand('insertText', true, message);
nextConversation();
previousConversation();
sendMessage();

if (previousMessage) {
  document.execCommand('insertText', true, previousMessage);
  nextConversation();
  previousConversation();
}

[JoniVR]

Yeah, I mean it's not the most elegant solution to use a deprecated API but I don't see any other way to currently get it done otherwise. So should I just implement it with your workaround then?

[CvX]

Yeah, I suppose so. We can always change it if we come up with a better one. 🙂

[JoniVR]

Alright, I'll take a stab at implementing it tomorrow, thanks for the help with this.

[JoniVR]

Hi @CvX
Quick question.
So now to move it to notifications-isolated.js I should use:

ipc.on('notification-reply', (event, data) => {
	const previousConversation = getIndex();
	window.postMessage({type: 'notification-reply', data}, '*');
	window.addEventListener('message', async ({data: {type}}) => {
		if (type === 'reply-callback') {
			await sendReply(data.reply);
			if (previousConversation) {
				selectConversation(previousConversation);
			}
		}
	});
});

inside browser.js
and post back a message back from notifications-isolated.js to the browser:

if (type === 'notification-reply') {
	window.postMessage({type: 'reply-callback'}, '*');
}

like that right? Or am I missing something?

[CvX]

Yeah, but you'll have to move the window.addEventListener out of the IPC handler (otherwise you'd be adding a new listener after each reply). 🙂

[JoniVR]

It's probably going to be an issue to switch to the previous conversation then, unless we send it back and forth through postMessage I guess.. or save it globally, but I think it'd be better to send it back and forth then.

[JoniVR]

I think it's about ready now as far as I can tell, could you do some testing of your own?

I'm asking some people to send me messages so I can try replying when notifications from multiple people are coming in. Might also want to check Windows and Linux again to see if anything is broken on that end.

edit: Yeah, I can't find any issues on my end, messages are sent to the correct people, emojis work, autoswitching back works, saving and restoring previous messages in the field work,... let me know if there's anything else you want me to look into.

[CvX]

I'm currently testing this. I've found some issues, but they're not directly caused by this PR:

1. You reply via notification. The contact responds. Messenger doesn't trigger a notification at all, probably because it assumes you're active, in the app. This makes you miss the message (though there's the unread badge on app's icon in the Dock) and you can't continue the conversation through notifications.
   It may be possible to fix this. Messenger reacts to some presence events fired by the browser (window.onblur, Page Visibility API, etc.), so we could trigger those after sending the reply.
2. There's a bug in Messenger, when you switch to a conversation with a bot, sometimes the message composer is missing:
   
   When it's gone, the reply obviously doesn't go through.
3. Sometimes, the Messenger does fire a notification, but due to a bug (most likely in Electron) it doesn't appear on the screen and goes straight to the Notification Center. I'm gonna look into it.
4. There are also cases where you type a reply, click send or hit enter, and then… nothing. The notification disappears but notification.on('reply') isn't called. Neither is notification.on('click') nor notification.on('close'). That's a bug in either Electron or macOS… I'm going to look deeper into this one too.

[JoniVR]

1. You reply via notification. The contact responds. Messenger doesn't trigger a notification at all, probably because it assumes you're active, in the app. This makes you miss the message (though there's the unread badge on app's icon in the Dock) and you can't continue the conversation through notifications.

I'm not having this issue, probably because I've disabled my active status by default. But it makes sense. I'll look into it.

2. There's a bug in Messenger, when you switch to a conversation with a bot, sometimes the message composer is missing.
   When it's gone, the reply obviously doesn't go through.

I don't think we can do much about that, but I also don't think people use bots that much (or at least won't reply to them through notifications that much, at least I don't)

3. Sometimes, the Messenger does fire a notification, but due to a bug (most likely in Electron) it doesn't appear on the screen and goes straight to the Notification Center. I'm gonna look into it.
4. There are also cases where you type a reply, click send or hit enter, and then… nothing. The notification disappears but notification.on('reply') isn't called. Neither is notification.on('click') nor notification.on('close'). That's a bug in either Electron or macOS… I'm going to look deeper into this one too.

I had noticed both of these issues during my testing with the notifications on several occasions during this PR, but I just thought it was due to me triggering a lot of notifications and Apple somehow limiting it to prevent spam, but what you're saying makes a lot of sense.

I'll look into fixing the first issue, nr 3, 4 are something I'll research too, but again, I'm not an expert at electron, so my experience with it is pretty limited.

[CvX]

Update on issue 4:

It's a bug in Electron. Turns out Notification objects are being prematurely garbage collected… so in many cases clicking, replying, or closing a notification doesn't trigger anything because the underlying Notification object (and its callbacks) is gone.
To fix this, Electron has to keep track of notifications that have event listeners, as stated in Notification Web API spec (I'll submit an issue on Electron's repo).

I've pushed a workaround to this branch.

About other issues:

1: Gotta fix that, will look into that next.
2: Still happens, but definitely is a non-blocker for this PR. Just a bit annoying when it comes to testing replies. 😉
3: I haven't seen it recently, should be easier to spot after 1 is fixed.

[JoniVR]

Hi @CvX

Any updates on this? Did you create an issue on electron (so I can track it myself)?
I resolved the merge conflicts after typescript migration.
I'm unable to reproduce issue 1. on my end at the moment.

[CvX]

@JoniVR Sorry, I've been away for a while. Here's the issue on the Electron repo: electron/electron#16922

About the issue "1.", the best way to reproduce it is to chat with a bot (I've created one to test this PR, but I'm waiting for it to go through FB's review)

• Send it a message
• Hide the window
• Reply to the first message
• Reply to the second message
• Reply to the third message
• Fourth message doesn't produce a notification

[JoniVR]

@CvX
Yes, I'll be waiting for that bot then, unless you can recommend me a different one?
I was previously testing it using the Memegenerator bot I saw from your screenshot earlier, but I think it has blocked me or something because my messages don't arrive anymore.

[CvX]

Yeah, Meme Generator either went down or it blocked us 😉 That's what prompted me to do my own bot (review still pending)

[CvX]

Alright, Messenger Ping Bot is live! 😄 Just search for it in the app and send anything.

[CvX]

I've pushed a fix: apparently the "Send" button lost one of its CSS classes recently. 😉

I was also investigating the issue with max 3 notifications. Turns out it's a "feature". 😉 Messenger tracks how many unread messages are in each thread and show notification only for the first 3 messages in each one. I haven't found any reliable way to work around it, so I'd say there's no reason to try to fight it and just accept it. 😉

There's a minor edge case that also comes to mind that I don't think I've mentioned before. Let's say you have unread messages in the currently active conversation. You reply via notification to a different conversation. But the first one is marked as seen/read even though you haven't.
Just something to keep in mind. I don't working around that is needed.

And with that, I think this PR is almost ready to be merged? 🙂
I haven't tested on Windows yet, but I'll be able to check that out tomorrow. 🚀

[JoniVR]

I was also investigating the issue with max 3 notifications. Turns out it's a "feature". 😉 Messenger tracks how many unread messages are in each thread and show notification only for the first 3 messages in each one. I haven't found any reliable way to work around it, so I'd say there's no reason to try to fight it and just accept it. 😉

Alright, makes sense, it was really weird that it did that anyways, couldn't really find a reason why it would happen.

There's a minor edge case that also comes to mind that I don't think I've mentioned before. Let's say you have unread messages in the currently active conversation. You reply via notification to a different conversation. But the first one is marked as seen/read even though you haven't.

Would it perhaps be possible to block the "seen indicator" in this case? The app already does this as a setting so perhaps that could be a workaround? I'm just guessing here.

Anyways, thanks a lot for all the help and feedback with this, I've been extremely busy lately so it's very helpful!

[CvX]

Would it perhaps be possible to block the "seen indicator" in this case? The app already does this as a setting so perhaps that could be a workaround? I'm just guessing here.

Yeah, that might be a good solution. 🙂 I'll try it later!
Anyway, that can be added later, after this PR is merged. 🙂

[JoniVR]

Sounds good to me, would be great to have this feature! Looking forward 👌🏻

[CvX]

Alright, no issues on Windows 10! 🚀

[sindresorhus]

Works great for me 👌

Thanks for implementing this, @JoniVR 🙌

[JoniVR]

Hi @sindresorhus

Is it possible that you forgot to add the other assets to the 2.29.0 release? Or is it normal that they don't show up yet?

Thanks for everything.
Best regards,
Joni

[sindresorhus]

Sometimes electron-builder fails to build the assets and I didn't notice. Fixed: https://github.com/sindresorhus/caprine/releases/tag/v2.29.0