Crash - Assertion failed: new_size % (1ULL << m_page_shift) == 0

[BlueCobold]

How frequently does the bug occur?

Always

Description

I'm trying to open a realm-file I have edited with RealmStudio and/or with RealmJava and/or compacted. It leads to the crash/exception below.
The file can be properly opened in RealmJava or with RealmStudio.
The file is encrypted, but I can provide both file and password via email.
This stops me from rolling out updates to my app.

Stacktrace & log output

/Users/realm/workspace/realm_realm-core_release_13.21.0/src/realm/util/encrypted_file_mapping.cpp:1028: [realm-core-13.21.0] Assertion failed: new_size % (1ULL << m_page_shift) == 0
0   Realm                               0x0000000106521868 _ZN5realm4utilL18terminate_internalERNSt3__118basic_stringstreamIcNS1_11char_traitsIcEENS1_9allocatorIcEEEE + 28
1   Realm                               0x00000001065217b0 _ZN5realm4util19terminate_with_infoEPKcS2_lS2_OSt16initializer_listINS0_9PrintableEE + 400
2   Realm                               0x0000000106521620 _ZN5realm4util19terminate_with_infoEPKcS2_lS2_OSt16initializer_listINS0_9PrintableEE + 0
3   Realm                               0x0000000106506494 _ZNSt3__16vectorIbNS_9allocatorIbEEE6resizeEmb + 0
4   Realm                               0x000000010650e414 _ZN5realm4util24extend_encrypted_mappingEPNS0_20EncryptedFileMappingEPvmmm + 196
5   Realm                               0x000000010650c360 _ZN5realm4util4File7MapBase13try_extend_toEm + 156
6   Realm                               0x000000010638d86c _ZN5realm9SlabAlloc18update_reader_viewEm + 664
7   Realm                               0x000000010638c33c _ZN5realm9SlabAlloc11attach_fileERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERNS0_6ConfigEPNS_4util13WriteObserverE + 1024
8   Realm                               0x00000001063c73c0 _ZN5realm2DB4openERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEbRKNS_9DBOptionsE + 1880
9   Realm                               0x00000001063cf814 _ZN5realm2DB6createENSt3__110unique_ptrINS_11ReplicationENS1_14default_deleteIS3_EEEERKNS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERKNS_9DBOptionsE + 164
10  Realm                               0x000000010662c10c _ZN5realm5_impl16RealmCoordinator7open_dbEv + 1184
11  Realm                               0x000000010662ce78 _ZN5realm5_impl16RealmCoordinator12do_get_realmEONS_11RealmConfigERNSt3__110shared_ptrINS_5RealmEEENS4_8optionalINS_9VersionIDEEERNS_4util17CheckedUniqueLockEb + 76
12  Realm                               0x000000010662cd68 _ZN5realm5_impl16RealmCoordinator9get_realmENS_11RealmConfigENSt3__18optionalINS_9VersionIDEEE + 460
13  Realm                               0x00000001066bbc98 _ZN5realm5Realm16get_shared_realmENS_11RealmConfigE + 120
14  Realm                               0x00000001062f5dc8 +[RLMRealm realmWithConfiguration:confinedTo:error:] + 892
15  Realm                               0x00000001062f5968 +[RLMRealm realmWithConfiguration:queue:error:] + 156
16  RealmSwift                          0x000000010520eb7c $sSo8RLMRealmC13configuration5queueABSo0A13ConfigurationC_So012OS_dispatch_C0CSgtKcfCTO + 128
17  RealmSwift                          0x00000001052afefc $s10RealmSwift0A0V13configuration5queueA2C13ConfigurationV_So012OS_dispatch_D0CSgtKcfC + 192
18  my-app-name                           0x000000010232cd50 $s9my-app-name7LibraryC7upgrade33_F15A9C9A19EBB745E05E9F5C484B9774LLyyFZ + 3804
19  my-app-name                           0x000000010232bdbc $s9my-app-name7LibraryC7preloadyyFZ + 1172
20  my-app-name                           0x000000010208d82c $s9my-app-name22TrackingViewControllerC10reloadData33_B717444B32E4031AA48693EEE6BC3E5DLLyyFyycfU_ + 312
21  Async                               0x00000001033f9f14 $s5Async0A5BlockV5async33_E687294E1917D2F98ED787A27C7576C6LL5after5block5queueACyytqd__GSdSg_qd__ycAA3GCDAELLOtlFZyycfU_ + 196
22  Async                               0x00000001033f9fe4 $sIeg_IeyB_TR + 48
23  libdispatch.dylib                   0x000000010471329c _dispatch_block_async_invoke2 + 104
24  libdispatch.dylib                   0x0000000104701d5c _dispatch_client_callout + 16
25  libdispatch.dylib                   0x0000000104716ae8 _dispatch_root_queue_drain + 1012
26  libdispatch.dylib                   0x000000010471753c _dispatch_worker_thread2 + 248
27  libsystem_pthread.dylib             0x00000001b059a83c _pthread_wqthread + 224
28  libsystem_pthread.dylib             0x00000001b059963c start_wqthread + 8

Can you reproduce the bug?

Always

Reproduction Steps

Whenever I try to open the file on Mac M1 using Simulator of iPhone 14 Pro running iOS 16.2 via Realm-Swift SDK

Version

10.42.x

What Atlas Services are you using?

Local Database only

Are you using encryption?

Yes

Platform OS and version(s)

iOS 16.2 and MacOS 13.4.1 (M1)

Build environment

ProductName:		macOS
ProductVersion:		13.4.1
ProductVersionExtra:	(c)
BuildVersion:		22F770820d

/Applications/Xcode.app/Contents/Developer
Xcode 14.2
Build version 14C18

/opt/homebrew/bin/pod
1.12.1
(not in use here)

/bin/bash
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin22)

carthage not found
(not in use here)

/usr/local/bin/git
git version 2.26.0

[sipersso]

Getting the same issue

[joaopdcgarcia]

same here. introducing a realm swift encrypted database with encrypted seed (both open via realm studio, not via app)
I'm on 10.42.4

[aaru]

Getting same issue..
It works fine on Simulator, but when deployed on device it crashes with above error at this line
let fileUrlIs = try schemaVersionAtURL(config.fileURL!, encryptionKey: keyData)
This was working perfectly fine on both device and simulator when I had realm-core of 13.6.0
But when I upgraded (10.43.1) to latest realm-core of 13.23.1 it started failing.
My realm file is encrypted database

[aaru]

After debugging with lot of trial and errors I found this bug started from v10.41.1 which has realm-core of 13.17.0 .
Culprit should be realm-core.
After downgrading swift-realm to 10.41.0 which has realm-core 13.15.1 this crash disappeared.
This crash happens only for encrypted db

[finnschiermer]

@BlueCobold Thx for reporting this. @aaru Thx for narrowing this down.

[finnschiermer]

We changed some stuff which could be related in v13.17.1 ... which leads me to this question:

@aaru Any chance you are wrong by one version and 10.41.1 (core 13.17.0) is actually OK while 10.42.0 (core 13.17.1) is the first one crashing?

[finnschiermer]

Also, if possible: could you tell me the exact size of the file at the point of failure?

[BlueCobold]

@finnschiermer I could provide you with a failing and a working version of the file - although one contains more data than the other and has a different file size.

[finnschiermer]

@BlueCobold that could also be useful - but only if you are fine with also supplying the encryption key?

but in any case, I may be able to deduce something from just the sizes of the files.

[BlueCobold]

@finnschiermer Providing the key would be alright as long as it's not publicly.
The broken file has a size of 2.277.376 byte.
The working file has a size of 2.281.472 byte.

[BlueCobold]

Any news to this? The problem still persists in 10.45.2. It does not occur in 10.41.1.

[nirinchev]

I'll move this issue to the realm-core repo as the bug appears to be Core-related.

[BlueCobold]

Is there any news to this? Without this fix, all new versions of Realm are not usable for productive use as they can lead to full data loss on the user's end.

[ironage]

Hey @BlueCobold would you still be willing to provide the file for us and the encryption key? If so please send them to [email protected] and reference this issue in your email. Having a repro would really help us track this down.
Other questions that may help narrow this down: is this a bundled Realm? Encryption uses the native device page size and so is not compatible for sharing between devices that have a different page size. If this is something that was working on earlier versions, then that's probably not the situation but worth checking.

[BlueCobold]

@ironage I guess I can provide an example file. What do you mean by "bundled Realm"? If you mean a realm-file which is being shipped within the app with initial data, then yes, that is the case and yes, it still works with earlier versions of realm while the newer versions don't. I don't know what's the page size of the devices in question, but I do assume that different Android and iPhone devices of course do have different page sizes and up until that version, sharing the realm file worked on any device and I am very dependent on this to work as users expect backups to also work in case they need to switch to a new or different device. Incompatibility is a no-go for a database in my eyes, because of backup- and device-migration use-cases which are daily business on mobile platforms.

[ironage]

@BlueCobold
Correct, by bundled realm I mean what you described; shipping a pre-seeded Realm with your app.
Sharing an encrypted Realm across devices with a different page size has never been supported (though non-encrypted Realms do not have this limitation). As you say, there is certainly valid use cases for supporting it and perhaps we can lift this limitation one day, but the limitation has not been imposed recently. Regardless, something has regressed for your use case in recent versions and it would really help us to have a repro to try and track it down; please provide an example Realm if you are able.

[safu9]

Hi, I created a repro project for this crash.
https://github.com/safu9/repro-realm/tree/6af4ec0d79f9e013ec2e754f291ec31de7f509c6

The repository has several realm files for testing in Database/Sources/Database/Resources.
You can try a different file by changing this line.
The files were created on Intel Mac instances of GitHub Actions instead of Realm Studio.

[BlueCobold]

@ironage I submitted the file in question via email.
PS: I accidentally wrote in my email that the issue appeared from version 10.41.1. This is incorrect, it does work in 10.41.1. It starts failing from 10.42.1 or 10.42.0.

[ironage]

Thanks @BlueCobold, we've received the file and I can reproduce the assertion on open. We will investigate.