FES reply with attachment is not decrypted

[sosnovsky]

Customer reported an issue when FES reply to password-protected message isn't decrypted by browser extension, when reply contains attached file with size between 500 and 800kb

Example message is in flowcrypt.compatibility account with subject FES message for #5458

[ioanmo226]

Hey @sosnovsky, I attempted to send a reply to an FES encrypted message that had a 600kb attachment. Everything looks good on my end.
However, the FES message for #5458 isn't decrypting.
How'd you create that message?

[sosnovsky]

It was created with the previous version of FES 2023-11, but in the latest 2023-12 release we updated FES to send messages in PGP/MIME format (https://github.com/FlowCrypt/enterprise-server/pull/5129) which fixed this issue.

Message in FES message for #5458 is still a valid encrypted message, but for some reason it's not detected as encrypted one. For this case when message content is clipped, browser extension is expected to fetch full message data from Gmail API (msgGetFull), but it's not triggered for mentioned message

[ioanmo226]

Yeah, I checked it and messages/{msgId}?format=full api returns below response. (Notice {body: {size: 0}} )

[ioanmo226]

Maybe need to fetch attachment data and proceed I think.
Will check deeply though.

[sosnovsky]

It happens because after some message size (that's why this issue is caused by attachments of specific size) Gmail puts all message content into body attachment. This case is also handled by browser extension, but for some reason it fails for this message

[ioanmo226]

I've noticed that the API returns a 200 status code after 3-5 seconds, which is why it's treated as plain text.
However, to get the actual response data, it takes about 100-200 seconds.
I'm not sure why this API takes so long to respond.

Should we look into optimizing the attachmentGetChunk function to speed it up? Even if we do, we'll still need to add a loading spinner to improve user experience, as people might think it's stuck or not working.

What are your suggestions for handling this?

https://www.googleapis.com/gmail/v1/users/me/messages/{msgId}/attachments/{attachmentId}

[sosnovsky]

Our goal here is to make encrypted message to be detected as encrypted and rendered appropriately.
As currently user won't be able to read message text, because it's detected as plain text.

Showing spinner is ok, but we should check why it takes so long to fetch message data here - for other messages it's much quicker.

Should we look into optimizing the attachmentGetChunk function to speed it up?

Do we use attachmentGetChunk here? As we also have fetchAttachment method, maybe it'll be faster

[ioanmo226]

It's not because of our implementation.
But messages/{msgId}/attachments/{attachmentId} api takes so long. It's google's problem I think

[ioanmo226]

@sosnovsky I sent you fetch code which will execute call to above api.
Run that code and it will take over 150 seconds to fetch data (even though final data is about 1.5mb or so)

[sosnovsky]

fetch code failed for me with CORS error. But I tried to perform this.messageRenderer.gmail.attachmentGet in replaceArmoredBlocks and message attachment was downloaded in a couple seconds.

I added code below before this line -

flowcrypt-browser/extension/js/content_scripts/webmail/gmail-element-replacer.ts

Line 168 in 5b7cf40

const msgId = this.determineMsgId(emailContainer);

const att = await this.messageRenderer.gmail.attachmentGet(
  '18b39cba715705cf',
  'ANGjdJ_39gxMSuHyMrzCNBPhG0PuxCH1web9tYiWMysHXvVADpZvHjFPQ-Zm9fXMzcjrEcnK8bRLDdUAzHUADzsRuUUuDxh5_FiRUc8lORnSk14CB56jXlethhBwAeXrk7V4OZOv2vfGAZ7rpH9pHg8thTBVWDvQjLFFQRH8GcmmLRwJbY2jr_wjyXTcsMZ-IEJXyv0TOlN8BpFmWIqxYiFr_4fl1ozObb30_sSULJb0QvUSM5Z-dibNRLjoANF_ywRLwQ0Dw7w90WU2Vya_oR3frxtJtilyU8Ca30evoxgpRW3EbmklQv4J7fVdoNrLcio6gHgObGGZ6jNSx7gVSu7Ws0aE7w9vJSoUMqdz2diZPNOZ8vF_5v6bo0KGoC0'
);
console.log(att.data.toUtfStr());

So Gmail API works fast, issue should be somewhere on our side

[ioanmo226]

Strange....
With above code, it fetches in a few seconds.
I wonder why simple fetch call take so long time (more than 100 seconds). To avoid CORS error you might try in our extension dev console

[ioanmo226]

Anyway, let me investigate and fix our logic.

[sosnovsky]

I wonder why simple fetch call take so long time (more than 100 seconds).

I noticed some issues with fetch too while working on #5372, especially on firefox, as it's implementation differs in chrome and firefox

[sosnovsky]

fixed in #5518