Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Background Crypto store usage #4370

Closed
wants to merge 11 commits into from
Closed

Conversation

ouchadam
Copy link
Contributor

@ouchadam ouchadam commented Oct 29, 2021

Fixes some parts of the crypto store flow being on the main thread, this will help address a bunch of RealmCryptoStore related ANRs as we'll no longer be calling Realm.getInstance on the main thread when interacting with the crypto store.

  • Fixed by lazily initialising any init logic
  • Moving view model init logic to Start events

This could break all sorts of things, needs a bunch more testing!

Before - Sign in
getOrCreateOlmAccount main
getDeviceTrackingStatuses Crypto_Thread
getPendingIncomingGossipingRequests main
getMyCrossSigningInfo main
getCrossSigningInfo main
open Crypto_Thread
areDeviceKeysUploaded Crypto_Thread
areDeviceKeysUploaded Crypto_Thread
getMyCrossSigningInfo Crypto_Thread
getCrossSigningInfo Crypto_Thread
getUserDevices Crypto_Thread
storeUserDevices Crypto_Thread
getCrossSigningInfo main
getUserDeviceList main
getCrossSigningInfo DefaultDispatcher-worker-6
getCrossSigningInfo DefaultDispatcher-worker-4
getCrossSigningPrivateKeys DefaultDispatcher-worker-9
getUserDeviceList main
getUserDeviceList DefaultDispatcher-worker-8
getMyDevicesInfo DefaultDispatcher-worker-5
getCrossSigningPrivateKeys DefaultDispatcher-worker-3
getUserDeviceList DefaultDispatcher-worker-6
setDeviceKeysUploaded Crypto_Thread
saveOlmAccount Crypto_Thread
saveMyDevicesInfo main
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
open Crypto_Thread
tidyUpDataBase Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
saveDeviceTrackingStatuses Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
saveDeviceTrackingStatuses Crypto_Thread
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
saveMyDevicesInfo main
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
getUserDevice Crypto_Thread
storeUserDevices Crypto_Thread
storeUserCrossSigningKeys Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getUserDevices Crypto_Thread
saveDeviceTrackingStatuses Crypto_Thread
getMyCrossSigningInfo DefaultDispatcher-worker-3
getCrossSigningInfo DefaultDispatcher-worker-3
getInboundGroupSession DefaultDispatcher-worker-3
getWithHeldMegolmSession DefaultDispatcher-worker-3
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
saveOlmAccount Crypto_Thread
getDeviceTrackingStatuses DefaultDispatcher-worker-3
saveOlmAccount Crypto_Thread
getMyCrossSigningInfo DefaultDispatcher-worker-3
getCrossSigningInfo DefaultDispatcher-worker-3
getInboundGroupSession DefaultDispatcher-worker-3
getWithHeldMegolmSession DefaultDispatcher-worker-3
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getMyCrossSigningInfo DefaultDispatcher-worker-3
getCrossSigningInfo DefaultDispatcher-worker-3
saveOlmAccount Crypto_Thread
getInboundGroupSession DefaultDispatcher-worker-3
getWithHeldMegolmSession DefaultDispatcher-worker-3
saveOlmAccount Crypto_Thread
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
shouldEncryptForInvitedMembers Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
getMyCrossSigningInfo DefaultDispatcher-worker-3
getCrossSigningInfo DefaultDispatcher-worker-3
getInboundGroupSession DefaultDispatcher-worker-3
getWithHeldMegolmSession DefaultDispatcher-worker-3
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
shouldEncryptForInvitedMembers Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
shouldEncryptForInvitedMembers DefaultDispatcher-worker-3
getDeviceTrackingStatuses DefaultDispatcher-worker-3
saveOlmAccount Crypto_Thread
getDeviceTrackingStatuses DefaultDispatcher-worker-3
getMyCrossSigningInfo DefaultDispatcher-worker-3
getCrossSigningInfo DefaultDispatcher-worker-3
getInboundGroupSession DefaultDispatcher-worker-3
getWithHeldMegolmSession DefaultDispatcher-worker-3
setKeyBackupVersion main
setKeysBackupData main
resetBackupMarkers main
getDeviceTrackingStatuses Crypto_Thread
getCrossSigningPrivateKeys DefaultDispatcher-worker-13
getDeviceTrackingStatuses Crypto_Thread
saveMyDevicesInfo main
After - Sign in
getDeviceTrackingStatuses Crypto_Thread
ensureCryptoMetadataEntity Crypto_Thread
open Crypto_Thread
areDeviceKeysUploaded Crypto_Thread
areDeviceKeysUploaded Crypto_Thread
getOrCreateOlmAccount Crypto_Thread
getMyCrossSigningInfo Crypto_Thread
getCrossSigningInfo Crypto_Thread
getUserDevices Crypto_Thread
storeUserDevices Crypto_Thread
getCrossSigningInfo DefaultDispatcher-worker-17
getUserDeviceList DefaultDispatcher-worker-17
getCrossSigningInfo DefaultDispatcher-worker-15
getCrossSigningPrivateKeys DefaultDispatcher-worker-14
getUserDeviceList DefaultDispatcher-worker-14
getCrossSigningInfo DefaultDispatcher-worker-5
getUserDeviceList DefaultDispatcher-worker-5
getCrossSigningPrivateKeys DefaultDispatcher-worker-15
getMyDevicesInfo DefaultDispatcher-worker-14
getUserDeviceList DefaultDispatcher-worker-8
saveMyDevicesInfo Crypto_Thread
setDeviceKeysUploaded Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
open Crypto_Thread
getPendingIncomingGossipingRequests Crypto_Thread
tidyUpDataBase Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
saveDeviceTrackingStatuses Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
saveDeviceTrackingStatuses Crypto_Thread
saveOlmAccount Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
saveOlmAccount Crypto_Thread
saveMyDevicesInfo Crypto_Thread
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getMyCrossSigningInfo DefaultDispatcher-worker-3
getCrossSigningInfo DefaultDispatcher-worker-3
getInboundGroupSession DefaultDispatcher-worker-3
getWithHeldMegolmSession DefaultDispatcher-worker-3
getUserDevice Crypto_Thread
storeUserDevices Crypto_Thread
storeUserCrossSigningKeys Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getUserDevices Crypto_Thread
saveDeviceTrackingStatuses Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
shouldEncryptForInvitedMembers Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getMyCrossSigningInfo DefaultDispatcher-worker-3
getCrossSigningInfo DefaultDispatcher-worker-3
getInboundGroupSession DefaultDispatcher-worker-3
getWithHeldMegolmSession DefaultDispatcher-worker-3
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getMyCrossSigningInfo DefaultDispatcher-worker-3
getCrossSigningInfo DefaultDispatcher-worker-3
getInboundGroupSession DefaultDispatcher-worker-3
getWithHeldMegolmSession DefaultDispatcher-worker-3
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getMyCrossSigningInfo DefaultDispatcher-worker-3
getCrossSigningInfo DefaultDispatcher-worker-3
getInboundGroupSession DefaultDispatcher-worker-3
getWithHeldMegolmSession DefaultDispatcher-worker-3
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
setShouldEncryptForInvitedMembers DefaultDispatcher-worker-3
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
shouldEncryptForInvitedMembers Crypto_Thread
getRoomAlgorithm Crypto_Thread
storeRoomAlgorithm Crypto_Thread
getCurrentOutboundGroupSessionForRoom Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getMyCrossSigningInfo DefaultDispatcher-worker-3
getCrossSigningInfo DefaultDispatcher-worker-3
getInboundGroupSession DefaultDispatcher-worker-3
getWithHeldMegolmSession DefaultDispatcher-worker-3
getDeviceTrackingStatuses Crypto_Thread
saveOlmAccount Crypto_Thread
getCrossSigningPrivateKeys DefaultDispatcher-worker-12
saveOlmAccount Crypto_Thread
getUserDevice Crypto_Thread
storeUserDevices Crypto_Thread
storeUserCrossSigningKeys Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
getUserDevices Crypto_Thread
saveDeviceTrackingStatuses Crypto_Thread
getMyCrossSigningInfo DefaultDispatcher-worker-13
getCrossSigningInfo DefaultDispatcher-worker-13
getCrossSigningPrivateKeys DefaultDispatcher-worker-8
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
getDeviceTrackingStatuses Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveOlmAccount Crypto_Thread
saveMyDevicesInfo Crypto_Thread
saveOlmAccount Crypto_Thread
setKeyBackupVersion Crypto_Thread
setKeysBackupData Crypto_Thread
resetBackupMarkers Crypto_Thread

@@ -226,7 +226,7 @@ internal class DefaultCryptoService @Inject constructor(
override fun fetchDevicesList(callback: MatrixCallback<DevicesListResponse>) {
getDevicesTask
.configureWith {
// this.executionThread = TaskThread.CRYPTO
this.callbackThread = TaskThread.CRYPTO
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by default the executor switches back to the main thread for the callback

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish #2449 get fully implemented one day...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's close! 😄

@github-actions
Copy link

github-actions bot commented Oct 29, 2021

Unit Test Results

  66 files  ±0    66 suites  ±0   59s ⏱️ -8s
135 tests ±0  135 ✔️ ±0  0 💤 ±0  0 ±0 
418 runs  ±0  418 ✔️ ±0  0 💤 ±0  0 ±0 

Results for commit 928f62e. ± Comparison against base commit 10ec6e7.

♻️ This comment has been updated with latest results.

Copy link
Member

@bmarty bmarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for cleanup up some of our init blocks.
A few remarks though.

@@ -226,7 +226,7 @@ internal class DefaultCryptoService @Inject constructor(
override fun fetchDevicesList(callback: MatrixCallback<DevicesListResponse>) {
getDevicesTask
.configureWith {
// this.executionThread = TaskThread.CRYPTO
this.callbackThread = TaskThread.CRYPTO
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish #2449 get fully implemented one day...

private val lazyRealmConfiguration by lazy {
Log.e("!!!", "ensureCryptoMetadataEntity ${Thread.currentThread().name}")
ensureCryptoMetadataEntity(realmConfiguration)
realmConfiguration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can we ensure that in the future realmConfiguration is not used directly by mistake? Maybe first add a comment?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmmmm we could create a LazyRealmConfiguration abstraction, what do you think?

also happy to add a comment (and remove the Log)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a comment on realmConfiguration to warn developer is enough... This code will be removed in a near future (Crypto Rust SDK).
And yes, please remove the log 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done d337b4f

observeInitialSync()
checkSessionPushIsOn()
observeCrossSigningReset()
override fun handle(action: HomeActivityViewActions) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diff is a bit strange here, at least from GitHub POW.
The method handle was already existing.
Also forks may suffer from this change, so it's probably better to ensure that they are minimal.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the diff looks really painful but all it's doing is moving the handle function to the top of the file and inlining the init functions into the ViewStarted action

I'll revert and leave the functions above the handle to improve the diff

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated including the started guard e354313

when (action) {
HomeActivityViewActions.PushPromptHasBeenReviewed -> vectorPreferences.setDidAskUserToEnableSessionPush()
HomeActivityViewActions.ViewStarted -> {
cleanupFiles()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously it was done only once, now it's done every time the Activity is created, i.e. also after a device rotation. Use isStarted pattern like in UnknownDeviceDetectorSharedViewModel?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, I'll guard all of the calls with isStarted to avoid any behaviour changes

@ouchadam ouchadam force-pushed the feature/adm/lazy-crypto-init branch 3 times, most recently from 6afa91e to 240ae9a Compare November 2, 2021 12:39
}
isStarted = true

viewModelScope.launch {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the diff looks big but the only change (apart from delaying the calls to startObserving) is moving the init logic into this viewModelScope.launch

@ouchadam ouchadam marked this pull request as ready for review November 10, 2021 08:59
@ouchadam
Copy link
Contributor Author

having been running this locally for 7+ days with no issues, promoting from draft

@bmarty
Copy link
Member

bmarty commented Nov 16, 2021

@ouchadam ktlint is not happy, there is a useless import to remove
@BillCarsonFr can you have a look on the PR?

Then I think we can merge it!

@bmarty bmarty requested a review from BillCarsonFr November 16, 2021 12:42
…ialise

- ensuring the crypto metadata is inserted doesn't take too long but the Realm.getInstance can be very heavy so we should delay where possible
…ocking work during object construction (init)
…itialisation tasks _after_ the Activity has started observing the events

- fixing any missing events which trigger before init and onCreate
- moves the cross sign off of the main thread
…han init

- also access the crypto devices off of the main thread
…alm crypto store on the main thread during the dagger graph creation
…y in the realm crypto store as we have the lazy version
@ouchadam ouchadam force-pushed the feature/adm/lazy-crypto-init branch from 0865710 to 928f62e Compare November 16, 2021 12:49
@ouchadam
Copy link
Contributor Author

ouchadam commented Nov 16, 2021

whilst rebasing I've found an issue with signout, am investigating

edit: the issue is present in develop #4480

@ouchadam
Copy link
Contributor Author

going to close, the crypto is being reworked with the rust SDK changes

@ouchadam ouchadam closed this Nov 26, 2021
@ouchadam ouchadam deleted the feature/adm/lazy-crypto-init branch February 25, 2022 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants