Consider storing megolm keys as an encrypted blob on the server

[richvdh]

I thought we had a bug to track this, but I can't find it. Storing the megolm keys on the server sounds a bit scary, but might present an interesting solution to bugs like element-hq/element-meta#647 and #3660

[richvdh]

obviously this raises the question of how to avoid your devices fighting with each other.

[pik]

This does sound scary, why not let users that really need this worry about doing it with their tool of choice (scp, syncthing, dropbox etc.) ?

[geez0x1]

This bug came from a discussion in the E2E room; me (and some others) were confused by the fact that exporting the E2E keys would not allow to view newer messages received after the export took place. This expectation - at least for me - stems from existing knowledge of public/private key crypto from e.g. GPG.

However, as Richard (richvdh) explained (correct me if I'm wrong at any point here):

1. Messages in rooms are not encrypted separately for every recipient like with GPG, but:
2. The room keys are evolving; after leaving a room future messages should not be decryptable, so future keys should not be derivable from current keys.
   As such, the Riot E2E room key export exports the room keys at that moment, allowing to decrypt only messages up to that point (and history up to the room settings?).

It seems desirable that the state (as Richard referred to it) should be recoverable in the case of a logout, browser crash, lost device, and so on, so that all received messages are accessible, all the time, as long as the exported secret is available (analog of the private key in the GPG scenario).

@pik: I am unsure how your suggestion would work?

[pik]

2. The room keys are evolving; after leaving a room future messages should not be decryptable, so future keys should not be derivable from current keys.

You can derive future keys from current keys as long as you have the message history for the room, unless they are rotated, in which case you would need to have someone share a more recent session key with you for that particular rotation.

It seems desirable that the state (as Richard referred to it) should be recoverable in the case of a logout, browser crash, lost device, and so on, so that all received messages are accessible, all the time, as long as the exported secret is available (analog of the private key in the GPG scenario).

Megolm messages should be recoverable on logout / lost-device if the session keys were exported afaik, because the InboundGroupSession is not ratcheted, though that only covers the current set of sessions? A full historical backup would need to store either old session keys or the entire decrypted history I guess? Either of those options are much better than storing private-keys (even if encrypted) on a remote-host though...

btw WhatsApp for example will lose all old conversations on a lost device, unless you were explicitly saving the decrypted conversations (e.g. to google drive).

[geez0x1]

You can derive future keys from current keys as long as you have the message history for the room, unless they are rotated, in which case you would need to have someone share a more recent session key with you for that particular rotation.

Sure but afaik key rotation happens fairly frequently exactly for the reason that people who have left rooms should no longer be able to decrypt further messages in that room. Btw, isn't it a good idea to ensure that keys are always and immediately rotated when someone leaves the room?

Megolm messages should be recoverable on logout / lost-device if the session keys were exported afaik, because the InboundGroupSession is not ratcheted, though that only covers the current set of sessions? A full historical backup would need to store either old session keys or the entire decrypted history I guess? Either of those options are much better than storing private-keys (even if encrypted) on a remote-host though...

I suppose storing old session keys would make most sense. And yes, that works, up to the moment the keys were exported like mentioned before. When you say "current set of sessions", you mean that it only allows to recover the contents of rooms the user is currently a member of?

I feel quite strong about not storing private keys remotely as well, but the current state in my opinion is very user unfriendly. Matrix/riot is extremely well suited for situations where email is great but too slow, e.g. working/team environments, kind of like Slack I suppose (which I don't use). However in those situations, looking back at older messages is frequent and it would be great to ensure history can always be recovered if desired.

In the case of email/GPG, one can simply keep the private key somewhere safe and ensure the email server is redundant and backed up, and messages will never ever be lost. This works for GPG due to its architecture, but not here. Here, currently, I need to somehow rely on my browser not to lose the localstorage contents and/or rely on external scripts that ensure I don't lose them.

WhatsApp, Signal, Telegram and others have similar drawbacks like you mention; the keys are not exportable at all afaik.

I do not have a real solution to this, and I'm not sure one exists. I suppose client software can solve this, by ensuring all keys are kept available perpetually and written to some local file (which the user can then back up - continuously).

A source of redundancy are the other users in a room, they could potentially share the room's history. Does the "Who can read history?" -> "Members only (since the point in time of selecting this option)" option play nice with E2E if I add a new device?

[ara4n]

https://github.com/vector-im/riot-web/issues/2286#issuecomment-299532155 contains some notes from discussing this with rich & mark.

[ara4n]

proposal for this is at matrix-org/matrix-spec-proposals#1219

[ara4n]

We now have the primitives for this in olm thanks to @uhoreg, who is taking a look at resurrecting my (very WIP) branches for this at matrix-org/matrix-js-sdk#595 and matrix-org/matrix-react-sdk#1703 and https://github.com/matrix-org/synapse/tree/matthew/e2e_backups

[ara4n]

(this has risen up the todo list because of various folk wanting to turn on E2E by default on deployments where users only have a single device, so not having a live encrypted backup of the megolm keys is potential disaster if the device goes missing).

[dbkr]

Current status of this:

• Making a backup a restoring it works
• Riot will look for a backup & see whether it can trust it, and if it can, use it

Remaining problems:

• The server re-uses versions from deleted backups which means if you delete a backup you get into a screwed state because the server will keep the old versions of the keys encrypted with the old key (E2E Key Backup: The server re-uses versions from deleted backups which means if you delete a backup you get into a broken state because the server will keep the old versions of the keys encrypted with the old key #7448)
• Keys are sent to the server when a room key event arrives and then never retried, also failing the message send if it fails (E2E Key Backup: Keys are sent to the server when a room key event arrives and then never retried, also failing the message send if it fails #7447)
• Olm.PkEncryption / Olm.PkDecryption objects aren't freeed
• Recovery key is a pickle of the Olm PkEncryption object which is several time longer than it needs to be (E2E Key Backup: Recovery key is a pickle of the Olm PkEncryption object which is several time longer than it needs to be #7446)
• If the backup is signed by this device, say so explicitly
• Sensible error needed when you enter an invalid recovery key

[Mikaela]

Was this forgotten as open or is there still something missing?

[ara4n]

more that we haven’t finished triaging since releasing 1.0. this is indeed done.