Snapshot stops working after Capacitor app being minimized for 5m

[vojto]

Operating System

iOS 17.4

Browser Version

Safari 604.1

Firebase SDK Version

10.9.0, 10.8.1

Firebase SDK Product:

Firestore

Describe your project's tooling

React, Next.js, Capacitor

Describe the problem

Over the months, we noticed some errors were always logged in Sentry, such as:

1. UnknownError: Attempt to get a record from database without an in-progress transaction
2. UnknownError: Attempt to get all index records from database without an in-progress transaction
3. UnknownError: Connection to Indexed Database server lost. Refresh the page to try again
4. FIRESTORE (10.7.0) INTERNAL ASSERTION FAILED: Unexpected state
5. Non-Error promise rejection captured with value: null

We always saw them, but never received any bug reports about broken sync, so we just shrugged them off as recoverable.

iOS 17.4

Since latest iOS, we noticed that sync stopped working sometimes. After focusing on this problem, it turned out it's likely to fail if app is running in the background, and is brought to foreground.

I want to stress that we are handling onError in our snapshot listeners, and restart the subscriptions in such case. However, the onError callback was never getting called, it would just stop working.

Coming back to app running in the background would often look like this:

So it shows the error, and snapshots are not working anymore.

Previous versions of iOS

To be clear, the errors above were also logged pre-iOS 17.4. But we didn't receive reports of broken sync before, and the errors were also less frequent.

For example, the "Connection to Indexed Database server lost" error is now most common on 17.4 after only few days that it's been out:

Logged errors

Whenever I encountered a failure, it always came with the INTERNAL ASSERTION FAILED error.

Sometimes, I logged other errors mentioned at the top - while app was still in the background.

Logging

I wanted to capture more information, but strangely whenever I set logging to firestore.setLogLevel('debug'), I couldn't repro once. And I tried many times.

After switching back to firestore.setLogLevel('error'), I was able to repro again.

iPhone state

I haven't noticed anything special about the state - things such as battery %, battery saving mode don't seem to have any effect. Also it didn't matter whether I flooded memory with other apps or not. (Obviously I didn't flood the memory enough for app to be terminated.)

Firebase versions

Reproduced on both 10.8.1 and 10.7.0.

Steps and code to reproduce issue

We don't have a repo with clean reproduction, but I want to explain how we can reproduce it in our app.

1. Run the Capacitor app on a phone
2. Switch to other apps. Use the device normally, but don't reopen the app
3. Wait 5 minuts
4. Make changes on another device
5. Reopen the app. The changes aren't showing up, and the FIRESTORE (10.7.0) INTERNAL ASSERTION FAILED: Unexpected state is thrown at the time when app is reopened.

[vojto]

cc @maccman

[vojto]

Tested on 10.9.0, not fixed.

[vojto]

I can consistently reproduce it by just:

1. Open the app
2. Switch to another app, let it run in the background
3. After ~3m, it'll throw INTERNAL ASSERTION FAILED and sync won't work after that

If I keep the app in the foreground, everything works fine.

Here's what the error looks like from remote Safari inspector:

[ehsannas]

Thanks for reporting @vojto . Can you point me to the repo that I can clone to reproduce?

[vojto]

@ehsannas Sadly we don't have a repo that reproduces the problem. It happens in our app where we use IndexedDb persistence.

[vojto]

Here is a Loom showing the stacktrace for this error: https://www.loom.com/share/995aa2184fb04e62a5081fefbbced2b3

Also this looks a lot like this old bug: https://stackoverflow.com/questions/56496296/how-do-i-fix-firestore-sdk-hitting-an-internal-error-was-encountered-in-the-ind

@ehsannas any clue?

[ehsannas]

Thanks for the Loom, @vojto. It appears to be an IndexedDB bug. We could try to confirm this by using a different iOS version [perhaps using the iOS simulator?]

[vojto]

@ehsannas So we've managed to reproduce the problem in a separate project: https://github.com/vojto/trips The description contains steps to reproduce, and also a Loom video showing how to do it.

We also noticed that removing Sentry fixes the problem.

Our Sentry setup looks like this - there's @sentry/nextjs and @sentry/capacitor - so Sentry is actually running as Swift code on the native side, and JS side is only sending events to it.

I want to note that Sentry has an option to use IndexedDb for errors collected while offline, but we are not using this option. So Sentry in our setup doesn't use IndexedDb.

We found out about 3 ways to fix the error:

1. Remove Sentry entirely
2. Remove @sentry/capacitor and only use @sentry/nextjs
3. Disable breadcrumbs (!)

Disabling breadcrumbs

Sentry tracks each fetch request, and each one is being sent to native side:

There's a simple way to disable this, just configure Sentry like this:

import * as Sentry from "@sentry/capacitor";
import * as SentryNext from "@sentry/nextjs";

let dsn = "https://[email protected]/0";
Sentry.init(
  {
    dsn,
    beforeBreadcrumb: () => {
      return null;
    },
  },
  SentryNext.init
);

When we do this, the error seems resolved.

---

My hunch is that this is somehow related to logging. When I tried switching to debug log level, I could not reproduce the error.

It would be great if you could look into why was this happening in the first place. How is it possible that Sentry capturing breadcrumbs causes Firebase to completely fail?

Let us know if we can help figuring out more.

[luke-rogers]

We also see the same issue with our Capacitor app running Firebase 9.x release.

Seems like the only way forward is to disable the breadcrumbs for now, not ideal but better than complete removal of Sentry.

[luke-rogers]

@vojto don't suppose you have any further information on this?

We just upgraded to the latest Firebase version (10.10.0) and disabled the breadcrumbs but are still seeing this issue. We are going to try removing @sentry/capacitor next and just use @sentry/angular-ivy.

[luke-rogers]

Removing @sentry/capacitor and just using @sentry/angular-ivy and disabling the breadcrumbs certainly reduces the occurrence of the issue but we are still seeing it for most iOS users.

@vojto did that completely resolve the issue for you or are you still seeing it reported?

[vojto]

@luke-rogers I'm seeing even more reports in Sentry now.

We added handling that will print message "Sync paused" to users whenever it happens. Users then have an option to refresh the entire app. No one reported seeing this banner lately, so I'm assuming it's not happening as much? Hard to tell though.

I'm also not able to reproduce my original issue - when I go into background, and come back 5m later, the sync still works, and no error is reported.

We're now focusing our efforts on moving away from Firestore Offline IndexedDb, and building our own offline-first database. We only want to use Firebase for storage and live sync of changes, but we wanna do our own storage - where Capacitor is a first class citizen.

[alexookah]

I have also capacitor with firebase installed.
I dont have sentry installed, But I also experience this issue opening the app from background app does not get new data.

[braincomb]

We are seeing this issue too. Sometimes there's no issues whatsoever having the app backgrounded overnight, but other times listeners completely stop working after 10 minutes or so of having the app backgrounded, so the app needs to be restarted.

Running latest Firebase JS SDK v10.13.0.

[madmacc]

Also regularly experiencing these issues in the Sentry logs for iOS.
I was hoping I could just ignore them but based on the comments above it seems they may actually be giving the users issues.
I don't have @sentry/capacitor (although this is a Capacitor project) have just installed @sentry/angular-ivy
I will try removing breadcrumbs.

Firebase: JS SDK v8.2.5

[madmacc]

Also regularly experiencing these issues in the Sentry logs for iOS. I was hoping I could just ignore them but based on the comments above it seems they may actually be giving the users issues. I don't have @sentry/capacitor (although this is a Capacitor project) have just installed @sentry/angular-ivy I will try removing breadcrumbs.

Firebase: JS SDK v8.2.5

Removing breadcrumbs did not fix the issue.