Seshat stopped working for me in Element Desktop 1.9.0 #102

dkasak · 2021-09-29T17:29:31Z

Seshat stopped working for me inside Element Desktop 1.9.0 with the following message:

Error opening the database: SqlCipherError("Sqlcipher support is missing")

Which is weird because sqlcipher is installed and I haven't changed my setup. I'm not sure when it stopped working and I'm failing to trace it to a particular update.

The problem seems to be that this query returns 0 results. This then makes this check fail. Recompiling element-desktop with that check disabled makes seshat work again (though it triggered a reindexing for some reason).

poljar · 2021-09-30T11:13:29Z

The problem seems to be that this query returns 0 results. This then makes this check fail. Recompiling element-desktop with that check disabled makes seshat work again (though it triggered a reindexing for some reason).

The query is a valid check for sqlcipher support, you just disabled encryption for your event index. The reindex was presumably triggered because a fresh empty and most importantly unencrypted database was created.

We have tests that check for this functionality in the repo, also you can check if this works correctly using node itself:

$ node
Welcome to Node.js v16.10.0.
Type ".help" for more information.
> const {Seshat} = require('./');
undefined
> const db = new Seshat("path_to_my_store", {passphrase: "MY_PASSPHRASE"});

Adding debug prints for the sqlcipher support yields an additional line when opening the store:

$ node
Welcome to Node.js v16.10.0.
Type ".help" for more information.
> const {Seshat} = require('./');
undefined
> const db = new Seshat("path_to_my_store", {passphrase: "MY_PASSPHRASE"});
sqlcipher check: [Ok("4.4.3 community")]

In other words, the check works fine. Removing the check is not a fix, nor workaround. It just turns a broken setup into a silently broken one.

We can leave the the debug line in and try to run the same Seshat binary through Element, this will yield the line on the terminal from which Element is being executed:

sqlcipher check: []

So SQLCipher support seems to be missing, which is strange considering that libsqlcipher is linked into the Seshat library:

$ ldd node_modules/matrix-seshat/native/index.node
	linux-vdso.so.1 (0x00007fffceddd000)
	libsqlcipher.so.0 => /usr/lib/libsqlcipher.so.0 (0x00007fc706bea000)
  ...

Furthermore, when starting up, we can see that Element finds libsqlcipher:

$ strace electron15 . --profile-dir ./profile_dir
...
openat(AT_FDCWD, "/usr/lib/libsqlcipher.so.0", O_RDONLY|O_CLOEXEC) = 41
...

Considering that all Seshat tests still pass, the check worked for around ~1 year correctly, no code changes were made in Seshat, and most importantly Seshat works from node, I would say that this seems to be an Electron bug or some other strange interaction with packaging Element.

That being said, not sure how to debug this further.

dkasak · 2021-09-30T12:32:07Z

The query is a valid check for sqlcipher support, you just disabled encryption for your event index. The reindex was presumably triggered because a fresh empty and most importantly unencrypted database was created.

Right, the new store is unencrypted.

We have tests that check for this functionality in the repo, also you can check if this works correctly using node itself:

Running that code results in:

const db = new Seshat("/home/dkasak/.config/Element.bak/EventStore/events.db", MY_PASS);
Uncaught TypeError: failed to downcast any to object
    at new Seshat (/home/dkasak/code/seshat/seshat-node/lib/index.js:171:39)

So SQLCipher support seems to be missing, which is strange considering that libsqlcipher is linked into the Seshat library:

Yep, can confirm that seshat is linked properly and the library is correctly installed:

❯ ldd native/index.node
	linux-vdso.so.1 (0x00007ffd9d9d6000)
	libsqlcipher.so.0 => /usr/lib/libsqlcipher.so.0 (0x00007f826d210000)
	libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f826d209000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f826d1ee000)
	librt.so.1 => /usr/lib/librt.so.1 (0x00007f826d1e3000)
	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f826d1c2000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007f826d07e000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f826ceb0000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007f826d850000)
	libz.so.1 => /usr/lib/libz.so.1 (0x00007f826ce96000)
	libcrypto.so.1.1 => /usr/lib/libcrypto.so.1.1 (0x00007f826cbb7000)

❯ pacman -Qo /usr/lib/libsqlcipher.so.0
/usr/lib/libsqlcipher.so.0 is owned by sqlcipher 4.4.3-1

Furthermore, when starting up, we can see that Element finds libsqlcipher:

Same for me:

❮ strace electron13 . --profile-dir ./profile_dir |& grep cipher       
openat(AT_FDCWD, "/usr/lib/libsqlcipher.so.0", O_RDONLY|O_CLOEXEC) = 39

Considering that all Seshat tests still pass

Good thinking, I forgot to test whether tests are passing and there are some failures for me:

---- database::database_upgrade_v1_2 stdout ----
thread 'database::database_upgrade_v1_2' panicked at 'Database doesn't need a reindex.', src/database/mod.rs:1006:18
stack backtrace:
   0: std::panicking::begin_panic
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/std/src/panicking.rs:543:12
   1: seshat::database::database_upgrade_v1_2
             at ./src/database/mod.rs:1006:18
   2: seshat::database::database_upgrade_v1_2::{{closure}}
             at ./src/database/mod.rs:998:1
   3: core::ops::function::FnOnce::call_once
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/core/src/ops/function.rs:227:5
   4: core::ops::function::FnOnce::call_once
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- database::recovery::test::test_recovery stdout ----
thread 'database::recovery::test::test_recovery' panicked at 'Database doesn't need a reindex.', src/database/recovery.rs:411:22
stack backtrace:
   0: std::panicking::begin_panic
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/std/src/panicking.rs:543:12
   1: seshat::database::recovery::test::test_recovery
             at ./src/database/recovery.rs:411:22
   2: seshat::database::recovery::test::test_recovery::{{closure}}
             at ./src/database/recovery.rs:402:5
   3: core::ops::function::FnOnce::call_once
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/core/src/ops/function.rs:227:5
   4: core::ops::function::FnOnce::call_once
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- index::paginated_search stdout ----
thread 'index::paginated_search' panicked at 'attempted to leave type `linked_hash_map::Node<uuid::Uuid, index::Search>` uninitialized, which is invalid', /home/dkasak/.local/share/cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/linked-hash-map-0.5.2/src/lib.rs:174:52
stack backtrace:
   0: rust_begin_unwind
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/std/src/panicking.rs:517:5
   1: core::panicking::panic_fmt
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/core/src/panicking.rs:93:14
   2: core::panicking::panic
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/core/src/panicking.rs:50:5
   3: core::mem::uninitialized
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/core/src/mem/mod.rs:665:9
   4: linked_hash_map::LinkedHashMap<K,V,S>::ensure_guard_node
             at /home/dkasak/.local/share/cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/linked-hash-map-0.5.2/src/lib.rs:174:52
   5: linked_hash_map::LinkedHashMap<K,V,S>::insert
             at /home/dkasak/.local/share/cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/linked-hash-map-0.5.2/src/lib.rs:304:9
   6: lru_cache::LruCache<K,V,S>::insert
             at /home/dkasak/.local/share/cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/lru-cache-0.1.2/src/lib.rs:119:23
   7: seshat::index::IndexSearcher::search
             at ./src/index/mod.rs:379:13
   8: seshat::index::paginated_search
             at ./src/index/mod.rs:721:24
   9: seshat::index::paginated_search::{{closure}}
             at ./src/index/mod.rs:708:1
  10: core::ops::function::FnOnce::call_once
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/core/src/ops/function.rs:227:5
  11: core::ops::function::FnOnce::call_once
             at /rustc/0035d9dcecee49d1f7349932bfa52c05a6f83641/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    database::database_upgrade_v1_2
    database::recovery::test::test_recovery
    index::paginated_search

dkasak · 2021-09-30T12:35:39Z

Notably, opening the database manually with sqlcipher works:

❯ sqlcipher .config/Element.bak/EventStore.bak/events.db
SQLite version 3.34.1 2021-01-20 14:10:07 (SQLCipher 4.4.3 community)
Enter ".help" for usage hints.
sqlite> PRAGMA cipher_version;
4.4.3 community

It's also curious that this is happening for me but not for you with the same element-desktop package on Arch Linux. 🤔

poljar · 2021-09-30T13:29:35Z

Running that code results in:

const db = new Seshat("/home/dkasak/.config/Element.bak/EventStore/events.db", MY_PASS);
Uncaught TypeError: failed to downcast any to object
    at new Seshat (/home/dkasak/code/seshat/seshat-node/lib/index.js:171:39)

You didn't run the provided code, the second argument needs to be an object like it was shown in the example above:

const db = new Seshat("path_to_my_store", {passphrase: "MY_PASSPHRASE"});

Good thinking, I forgot to test whether tests are passing and there are some failures for me:

The tests are sadly stateful, remove/restore the ./data directory and they should pass.

Notably, opening the database manually with sqlcipher works:

❯ sqlcipher .config/Element.bak/EventStore.bak/events.db
SQLite version 3.34.1 2021-01-20 14:10:07 (SQLCipher 4.4.3 community)
Enter ".help" for usage hints.
sqlite> PRAGMA cipher_version;
4.4.3 community

Yes, opening the store seems to work from Seshat and sqlcipher, unless you try to open it from inside of Electron.

It's also curious that this is happening for me but not for you with the same element-desktop package on Arch Linux. thinking

It is happening for me, I already mentioned that I'm reproducing the issue from within Element, this line:

We can leave the the debug line in and try to run the same Seshat binary through Element, this will yield the line on the terminal from which Element is being executed:

sqlcipher check: []

dkasak · 2021-09-30T13:57:49Z

const db = new Seshat("path_to_my_store", {passphrase: "MY_PASSPHRASE"});

Ah right, sorry. It indeed loads correctly with that change.

The tests are sadly stateful, remove/restore the ./data directory and they should pass.

True, and the two stateful tests do indeed fail on subsequent runs, but this error happens on the first too:

thread 'index::paginated_search' panicked at 'attempted to leave type `linked_hash_map::Node<uuid::Uuid, index::Search>` uninitialized, which is invalid', /rustc/c8dfcfe046a7680554bf4eb612bad840e7631c4b/library/core/src/mem/mod.rs:663:9

poljar · 2021-09-30T16:08:55Z

True, and the two stateful tests do indeed fail on subsequent runs, but this error happens on the first too:

Ah ok, that was an error in some dependency of rusqlite, element-hq/element-meta#1119 solves this but the original issue still isn't solved.

dkasak · 2021-10-14T12:31:43Z

Could be related to https://bugs.archlinux.org/task/69980. That issue is basically due to GTK3 deciding to link sqlite3 into the process which prevents the process from also linking to sqlcipher since they both use the same symbols.

Also see:

I opened an upstream issue for Arch Linux. A workaround for Arch at least which makes Seshat work again is to force the use of sqlcipher for Electron by modifying /usr/bin/element-desktop to:

#!/bin/sh

LD_PRELOAD=/usr/lib/libsqlcipher.so exec electron13 /usr/lib/element/app.asar --disable-dev-mode "$@"

As to how this will be fixed properly, I'm not really sure.

ArchangeGabriel · 2021-11-26T20:37:48Z

So that’s how it ended up being fixed for Signal: signalapp/better-sqlite3@917a6f5

I don’t know where is the appropriate place for Element to fix that.

heftig · 2022-03-05T17:04:34Z

Also related: element-hq/element-desktop#691

Arch just added a patch that makes the Linux build of element-desktop also use a vendored SQLCipher, with linking flags to keep it from interacting with any SQLite from the system, similar to what was done in signalapp/better-sqlite3#3.

MountainX · 2022-04-12T12:11:08Z

Also related: vector-im/element-desktop#691

Arch just added a patch that makes the Linux build of element-desktop also use a vendored SQLCipher, with linking flags to keep it from interacting with any SQLite from the system, similar to what was done in signalapp/better-sqlite3#3.

I'm running version 1.10.8-2 on Arch (Repository: Community) with electron17 (17.4.0-1). I'm still getting these crashes if message search is enabled.

ArchangeGabriel · 2022-04-12T20:12:59Z

@MountainX Can you get backtraces for them? Cause currently you’re the first to report still having crashes to my knowledge. Maybe there is some broken state somewhere in your local configuration because of the previous way things worked, though I would not advise cleaning the config folder as this might result in messages lost.

MountainX · 2022-04-13T02:37:00Z

@MountainX Maybe there is some broken state somewhere in your local configuration because of the previous way things worked, though I would not advise cleaning the config folder as this might result in messages lost.

Before I found the workaround of disabling Message Search, I was so desperate for a solution that I deleted my ~/.config/Element folder (and ~/.config/Electron folder) multiple times. Didn't lose any messages, but it also does not help this issue.

Cause currently you’re the first to report still having crashes to my knowledge.

In the Element-Desktop room on Matrix, I saw others discussing this issue yesterday, which is what led me to report my issue. One user was on Arch, like me and also running the same versions as me, which are the latest packages in the Arch repos.. At least one of the devs in the room seemed to be aware that the issue was still not full resolved, if I was following the conversation correctly.

Can you get backtraces for them?

I'm not a developer. but I can try if it isn't that involved. What are the steps for getting a backtrace? I can reproduce the issue any time by enabling Message Search. (That's the same behavior I saw others report.) Element & Electron will crash soon after it is enabled.

HarHarLinks · 2022-04-21T12:35:03Z

I can confirm the issue persists with arch's element desktop 1.10.10-1. They're not immediate, I'm not sure what exactly triggers them, and I have been using the nightly without search (and without the patch) rather than the stable with supposedly fixed search, apart from ~once a month when I actually need the search. In these cases I have still noticed crashes even within just the half hour of running the app.
Just now I needed the search and am now commenting because I saw the issue element-hq/element-desktop#691 just got closed, but my element stable crashed within 5 minutes, just as it had synced and decrypted the new messages since last time. I restarted in a shell to get some log but for now it is still running and the search indexing has caught up completely.

Update: hasn't crashed for 5h. Seems to have to do with actually working the sync or indexing.

MountainX · 2022-04-22T07:38:53Z

I can confirm the issue persists with arch's element desktop 1.10.10-1.

Yes, I'm on the same version and I still have these crashes too when message search is enabled.

Polve · 2022-10-26T17:54:58Z

I'm not using arch linux so it's not an option for me, but thanks for the indications

dkasak · 2022-10-26T20:52:08Z

@Polve That workaround is not Arch-specific: you should still have an element-desktop script starting the application. What matters is having that LD_PRELOAD=/usr/lib/libsqlcipher.so env variable exported in the environment when running it.

Polve · 2022-10-27T06:10:47Z

I don't have the indicated file in the variable LD_PRELOAD in my system, but I have /usr/lib/x86_64-linux-gnu/libsqlcipher.so.0.8.6, I can use it instead?

But where do I get the electron13 binary? I don't have /usr/lib/element/app.asar too.

Maybe the packaging is different?

zefir-git · 2023-02-27T05:31:05Z

Having same issue on Fedora 37 using https://download.opensuse.org/repositories/home:jejb1:Element/Fedora_37/home:jejb1:Element.repo

mandree · 2023-05-30T21:08:35Z

@williamd5 wrote:

Having same issue on Fedora 37 using https://download.opensuse.org/repositories/home:jejb1:Element/Fedora_37/home:jejb1:Element.repo

But the workaround is the same, on Fedora 38 I am using:
LD_PRELOAD=/lib64/libsqlcipher.so /usr/bin/element-desktop

securerootd · 2023-06-17T18:30:55Z

Having same issue on Fedora 37 using https://download.opensuse.org/repositories/home:jejb1:Element/Fedora_37/home:jejb1:Element.repo

I had exact same issue with Fedora 38 and same repo
Element version: 1.11.33
Olm version: 3.2.14

@williamd5 wrote:

Having same issue on Fedora 37 using https://download.opensuse.org/repositories/home:jejb1:Element/Fedora_37/home:jejb1:Element.repo

But the workaround is the same, on Fedora 38 I am using: LD_PRELOAD=/lib64/libsqlcipher.so /usr/bin/element-desktop

This works 💯

dkasak added the bug Something isn't working label Sep 29, 2021

FlorianFranzen mentioned this issue Sep 29, 2021

[Backport release-21.05] element-desktop: build native modules NixOS/nixpkgs#132633

Closed

1 task

nyanpasu64 mentioned this issue Oct 15, 2021

"Message search initialisation failed, check your settings for more information" element-hq/element-desktop#904

Closed

MrPrimate mentioned this issue Oct 19, 2021

SQLITE_NOTADB: Some Linux users cannot use SQLCIPHER MrPrimate/ddb-adventure-muncher#8

Closed

Skirmisher mentioned this issue Dec 4, 2021

Encrypted message search broken due to sqlcipher linking conflict (with workaround) taw00/element-rpm#51

Closed

austinbutler mentioned this issue Jan 2, 2022

element-desktop: fix "Sqlcipher support is missing" NixOS/nixpkgs#153279

Merged

13 tasks

davidmehren mentioned this issue Jan 7, 2022

Can't enable message search element-hq/element-desktop#829

Closed

dotlambda mentioned this issue Mar 5, 2022

Add -Bsymbolic linker flag to hide sqlite symbols deltachat/deltachat-node#539

Merged

t3chguy mentioned this issue May 26, 2022

Revert back to using libsqlcipher0 for Debian & Ubuntu packages of Desktop element-hq/element-desktop#367

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Seshat stopped working for me in Element Desktop 1.9.0 #102

Seshat stopped working for me in Element Desktop 1.9.0 #102

dkasak commented Sep 29, 2021

poljar commented Sep 30, 2021

dkasak commented Sep 30, 2021

dkasak commented Sep 30, 2021

poljar commented Sep 30, 2021 •

edited

Loading

dkasak commented Sep 30, 2021 •

edited

Loading

poljar commented Sep 30, 2021

dkasak commented Oct 14, 2021 •

edited

Loading

ArchangeGabriel commented Nov 26, 2021

heftig commented Mar 5, 2022 •

edited

Loading

MountainX commented Apr 12, 2022

ArchangeGabriel commented Apr 12, 2022

MountainX commented Apr 13, 2022

HarHarLinks commented Apr 21, 2022 •

edited

Loading

MountainX commented Apr 22, 2022

Polve commented Oct 26, 2022

dkasak commented Oct 26, 2022 •

edited

Loading

Polve commented Oct 27, 2022

zefir-git commented Feb 27, 2023

mandree commented May 30, 2023 •

edited

Loading

securerootd commented Jun 17, 2023 •

edited

Loading

Seshat stopped working for me in Element Desktop 1.9.0 #102

Seshat stopped working for me in Element Desktop 1.9.0 #102

Comments

dkasak commented Sep 29, 2021

poljar commented Sep 30, 2021

dkasak commented Sep 30, 2021

dkasak commented Sep 30, 2021

poljar commented Sep 30, 2021 • edited Loading

dkasak commented Sep 30, 2021 • edited Loading

poljar commented Sep 30, 2021

dkasak commented Oct 14, 2021 • edited Loading

ArchangeGabriel commented Nov 26, 2021

heftig commented Mar 5, 2022 • edited Loading

MountainX commented Apr 12, 2022

ArchangeGabriel commented Apr 12, 2022

MountainX commented Apr 13, 2022

HarHarLinks commented Apr 21, 2022 • edited Loading

MountainX commented Apr 22, 2022

Polve commented Oct 26, 2022

dkasak commented Oct 26, 2022 • edited Loading

Polve commented Oct 27, 2022

zefir-git commented Feb 27, 2023

mandree commented May 30, 2023 • edited Loading

securerootd commented Jun 17, 2023 • edited Loading

poljar commented Sep 30, 2021 •

edited

Loading

dkasak commented Sep 30, 2021 •

edited

Loading

dkasak commented Oct 14, 2021 •

edited

Loading

heftig commented Mar 5, 2022 •

edited

Loading

HarHarLinks commented Apr 21, 2022 •

edited

Loading

dkasak commented Oct 26, 2022 •

edited

Loading

mandree commented May 30, 2023 •

edited

Loading

securerootd commented Jun 17, 2023 •

edited

Loading