slow handling of huge database

[halset]

My app has around 2 GB of tiles loaded using sideloading. Some users might have a lot more, but this is a good average. After upgrading from ios-v5.4.0 to 6.2.0 to fix problem with sluggish map in iOS 14 (#411) the app is very slow to start and the first delete of a offline regions is very slow.

Steps to reproduce

1. App with ios-v5.4.0 and around 2GB of tiles loaded with sideloading.
2. Upgrade Mapbox iOS SDK to 6.2.0
3. Start app and try to delete a offline pack

Expected behavior

Expected that everything should work fine including a smooth map on iOS 14.

Actual behavior

The map is smooth on iOS 14, but the app is very slow in the following ways:

1. Starting the app for the first time after upgrading Mapbox iOS SDK takes around one minute. 89% of the time is used in mbgl::DatabaseFileSourceThread::DatabaseFileSourceThread and within that most of the time is spent in mbgl::OfflineDatabase::migrateToVersion7
2. Deleting the first offline pack is very slow and takes 3 minutes and 20 seconds. 97% of the time is used in mbgl::DatabaseFileSourceThread::DatabaseFileSourceThread doing mbgl::OfflineDatabase::vacuum(). Probably as it is setting PRAGMA auto_vacuum = INCREMENTAL and doing a VACUUM. We had a discussion on that in [core] Enable incremental vacuum for Offline DB mapbox-gl-native#15837
3. Starting the app after that are also very slow. According to Instruments, 11 seconds are used in mbgl::OfflineDatabase::setMaximumAmbientCacheSize(unsigned long long) at every boot of the application. This is with [[MGLOfflineStorage sharedOfflineStorage] setMaximumAmbientCacheSize:NSUIntegerMax ... in the apps viewDidLoad. Without setting the ambient cache size from the app, the startup is even slower as of a huge evict that takes more than 2 minutes.

Point 1 and 2 is one-time going to the new version that might be acceptable, but 3 is worse at it happens at every startup.

Earlier, I have maintained a version of Mapbox iOS SDK for this offline heavy use that did skip the ambient cache test and evict. However after #340, it has been harder/impossible(?) for externals to compile a version of Mapbox iOS SDK with changes to the core mapbox-gl-native library. So perhaps the best solution would be to try to backport #411 to a version earlier than #340.

Configuration

Mapbox SDK versions: Mapbox iOS SDK 6.2.0
iOS/macOS versions: iOS 14
Device/simulator models: iPhone X
Xcode version: 12.0

[julianrex]

cc @tmpsantos

[halset]

I now have a temp workaround. Backported #411 to the version I already used.

[halset]

My temp workaround is not good enough, so probably need to backport more to the old version. :/