General channel no longer visible after deletion, recreation

[holmesworcester]

Right now Quiet on Android is in a state where the general channel is not visible. That should never happen because even if the general channel is deleted it should be recreated.

On desktop the general channel was indeed recreated after deletion. But on Android I can no longer see it.

This is happening in the main quiet test community and I can provide access if necessary. I am not sure what exact sequence of steps caused the bug to happen, but what happened was that I deleted the general channel while the Android app was offline and when it came back online, the general channel was gone. It is still visible on desktop clients in the same community.

Is the list of channels in the sidebar eventually consistent in the backend? How do we ensure it converges? How do we ensure the frontend converges?

UPDATE: These are the channels I see in the frontend:

[holmesworcester]

Here was a useful tidbit from the logs. The local state is missing a channel from the database state.

03-11 07:56:02.921  4309  4309 I AndroidIME: KeyboardViewManager.updateKeyboardBodyViewHolderPaddingBottom():512 Set finalPaddingBottom = 26 while holderPaddingBottom = 26; navigationHeight = 126
03-11 07:56:02.994 10992 11036 I ReactNativeJS: { locallyStoredChannels: 
03-11 07:56:02.994 10992 11036 I ReactNativeJS:    [ '------_7ff9b3afb5c13de6f5e21306537dab01',
03-11 07:56:02.994 10992 11036 I ReactNativeJS:      'bugs_cc4c8f415f0f4ebeee04b7d3f2667175',
03-11 07:56:02.994 10992 11036 I ReactNativeJS:      'dev-talks_96ee2902c43101ac59b9653c82cce51a',
03-11 07:56:02.994 10992 11036 I ReactNativeJS:      'qa_70e7d127b20dbe8f1f851439c7416779',
03-11 07:56:02.994 10992 11036 I ReactNativeJS:      'random_d4447dc5d22f912827ddf786a106cb03' ],
03-11 07:56:02.994 10992 11036 I ReactNativeJS:   databaseStoredChannelsIds: 
03-11 07:56:02.994 10992 11036 I ReactNativeJS:    [ 'general_d01617283d88c8b65300fa08d4e82997',
03-11 07:56:02.994 10992 11036 I ReactNativeJS:      'qa_70e7d127b20dbe8f1f851439c7416779',
03-11 07:56:02.994 10992 11036 I ReactNativeJS:      'bugs_cc4c8f415f0f4ebeee04b7d3f2667175',
03-11 07:56:02.994 10992 11036 I ReactNativeJS:      'random_d4447dc5d22f912827ddf786a106cb03',
03-11 07:56:02.994 10992 11036 I ReactNativeJS:      'dev-talks_96ee2902c43101ac59b9653c82cce51a' ] }
03-11 07:56:03.004 10992 11001 W com.quietmobile: Cleared Reference was only reachable from finalizer (only reported once)

Do we update the local state from the database state on every return to this screen? Or whenever the app returns to foreground? If not we should! If so, what's breaking?

[EmiM]

Channels state is synchronized in those cases:

• on new channel replication
• when backend storage is being initialized
• when socket io connection is being initialized

[holmesworcester]

So it should be synchronizing on restart, correct? And it is not?

[EmiM]

I connected to the same community using Quiet 2.1.1 desktop on Linux. I was already a part of this community.
As soon as I connected to someone my application entered the invalid state - without general channel:

Informative parts of the logs:

// INITIAL STATE

backend:StorageService Channels names: [
  'qa_70e7d127b20dbe8f1f851439c7416779',
  '------_7ff9b3afb5c13de6f5e21306537dab01',
  'bugs_cc4c8f415f0f4ebeee04b7d3f2667175',
  'random_d4447dc5d22f912827ddf786a106cb03',
  'dev-talks_96ee2902c43101ac59b9653c82cce51a',
  'general_1bc8bf1b5c40106d87a92d711d050791'
] +0ms


backend:SocketService Awaiting readyness before emitting:  sendMessage +961ms
  backend:StorageService:err Could not send message. No 'general_1bc8bf1b5c40106d87a92d711d050791' channel in saved public channels +0ms


// AFTER CONNECTING TO ANOTHER PEER

backend:StorageService REPLICATED: Channels +5s
  backend:StorageService Replicated. /orbitdb/zdpuApYgaPUeVJEGnWkby37392G5aP3tj3J18CpsCVuw9ou8m/channels.random_d4447dc5d22f912827ddf786a106cb03 +8ms
  backend:StorageService Subscribed to channel qa_70e7d127b20dbe8f1f851439c7416779 +4ms
  backend:StorageService Subscribed to channel bugs_cc4c8f415f0f4ebeee04b7d3f2667175 +0ms
  backend:StorageService Subscribed to channel random_d4447dc5d22f912827ddf786a106cb03 +0ms
  backend:StorageService Subscribed to channel dev-talks_96ee2902c43101ac59b9653c82cce51a +0ms
creating channel
  backend:StorageService Creating channel general_d01617283d88c8b65300fa08d4e82997 +1ms
  backend:SocketService Awaiting readyness before emitting:  deleteChannel +51s
deleting channel storage {
  channelId: 'general_1bc8bf1b5c40106d87a92d711d050791',
  ownerPeerId: 'QmYYUsY9CQKojh3bUq1kUbRbpS4REKsHvn3EoVFvZQu3z1'
}
channel {
  id: 'general_d01617283d88c8b65300fa08d4e82997',
  name: 'general',
  owner: 'holmes',
  timestamp: 1709067588523,
  description: 'Welcome to #general'
}
  backend:StorageService Set general_d01617283d88c8b65300fa08d4e82997 to local channels +13ms
  backend:StorageService Created channel general_d01617283d88c8b65300fa08d4e82997 +2ms
  backend:StorageService Pubsub - subscribe to /orbitdb/zdpuAyCB2hhQLgqzhkbB4erH4B63tHaJ8EmpoArAeyyMRkS4T/channels.general_d01617283d88c8b65300fa08d4e82997 +0ms
  backend:StorageService Subscribing to channel  general_d01617283d88c8b65300fa08d4e82997 +0ms
  backend:SocketService Awaiting readyness before emitting:  deleteFilesFromChannel +18ms
backend:ConnectionsManagerService socketService - deleteFilesFromChannel {

(...) // DELETING FILES FROM general_1bc8bf1b5c40106d87a92d711d050791

}

backend:StorageService Subscribed to channel general_d01617283d88c8b65300fa08d4e82997 +0ms
  backend:StorageService deleteFilesFromChannel : isFileExist- true +1ms
  backend:ConnectionsManagerService Storage - channelDeletionResponse +33ms

(...)

backend:StorageService progress 1/21. Address: /orbitdb/zdpuAyCB2hhQLgqzhkbB4erH4B63tHaJ8EmpoArAeyyMRkS4T/channels.general_d01617283d88c8b65300fa08d4e82997 +570ms

[EmiM]

I don't know how app entered this state in the first place but I know why general channel is not being displayed.

channelsReplicatedSaga does the syncing.
First it removes channels from local storage that are not in the database, then updates local storage with channels that are in db (and not in local storage yet).

Database does not have channel ------. Someone probably deleted it at some point. Local storage still has it.
Database has channel general. Local storage does not have general.

quiet/packages/state-manager/src/sagas/publicChannels/channelsReplicated/channelsReplicated.saga.ts

Lines 27 to 53 in 5a5e0c5

	// Removing channels from store
	if (databaseStoredChannelsIds.length > 0) {
	for (const channelId of locallyStoredChannels) {
	if (!databaseStoredChannelsIds.includes(channelId)) {
	log(`Removing #${channelId} from store`)
	yield* put(publicChannelsActions.deleteChannel({ channelId }))
	yield* take(publicChannelsActions.completeChannelDeletion)
	}
	}
	}
	// Upserting channels to local storage
	for (const channel of databaseStoredChannels) {
	if (!locallyStoredChannels.includes(channel.id)) {
	log(`Adding #${channel.name} to store`)
	yield* put(
	publicChannelsActions.addChannel({
	channel,
	})
	)
	yield* put(
	messagesActions.addPublicChannelsMessagesBase({
	channelId: channel.id,
	})
	)
	}
	}

In this case channelsReplicatedSaga is stuck on the yield* take(publicChannelsActions.completeChannelDeletion) (waiting for whole deletion process to be completed) because deleteChannelSaga simply returns (no ack) when general channel in local storage is undefined.
Because of that general channel does not have a chance to be added to local storage and be visible on sidebar.

Imho this saga needs refactoring.

[EmiM]

Managed to reproduce it:

• Create community, add a few channels
• Join with second user, make sure that all channels are replicated
• Close second user app, make sure that backend-bundle process did not stay alive (see Backend process is still working even when Quiet app is turned off #2294)
• Delete one of regular channels and general channel
• Open second user app

[kingalg]

Version: 2.2.0-alpha.2

I followed the steps mentioned by Emilia and played around with this functionality a little more trying to break it, and I didn't manage to do it. I consider this issue fixed.