Signal 11 crash still present after SIMCTL_CHILD_NSZombieEnabled=YES + latest Firebase version #3207
Comments
|
Hi @shamilovtim, thanks for the report! Try replacing Also, note that the flag that we pass as a fix for the Firebase-related Signal 11 deallocation errors is |
|
Hey @asafkorem! Just confirmed that we do pass I know that you're passing the more specific flag inside of Detox itself but I was hoping that the Zombie flag might deal with any other method swizzling related crashes. Unfortunately the crashes were still present so the more general Zombie flag did not work. Passing no flag doesn't work either. |
shamilovtim
changed the title
|
@shamilovtim ah okay, I see.. |
No matter what version of Detox or whether or not we apply |
|
@shamilovtim this sounds like the current Signal 11 errors are more Expo related (and aren't coming from Firebase), based on some previous reports and my gut feeling... Also, can you tell me if you have some other dependency that uses swizzling in iOS? That would be hard to investigate based on the logs only, so we must have some project I can work with that reproduces this issue. |
|
@asafkorem In my opinion the suspects are: Expo (we don't use expo updates), Stripe (v14 stripe-ios via tipsi stripe), Bugsnag, Codepush, Segment Analytics Stripe iOS (v14) for sure has method swizzling. I was able to find it by scanning the source code of the Pod. Flipper uses method swizzling but will not be active during Release builds. (However, I do wonder if Flipper would crash detox debug)? Excuse my ignorance but is there any way of making Detox play nice with any and all method swizzling rather than the Firebase workaround created in the Google repo? |
|
@shamilovtim the problem in Firebase/Performance wasn't in Detox end but in their code (it pretty much assumed that no other framework does dynamic ISA-Swizzling for iOS classes), so it's pretty hard to assume that there is a general solution to the problem from Detox' code. However, once we'll finish with Detox-iOS transition to XCUITest framework, we may be able to avoid doing ISA swizzling for iOS classes so hopefully it will resolve such issues. |
These definitely sound suspicious 😅 It will be very helpful for us if you'll run DetoxTemplate (or any example RN project) with Detox tests and any of your dependencies that might cause this error, and try to reproduce the bug. I know it might take a lot of time, but it is also hard for me to reproduce this issue with other libraries or frameworks I'm not familiar with. |
|
We're seeing this, I can maybe narrow down a bit as we intersect with some of those dependencies. We're using bits of Expo and CodePush. We also use |
|
We can't really nail down the cause. What version |
|
We're running |
|
We're on |
|
Possibly, although I can't help but feel it might be animation-timing related given the places we see this are always slightly after an animation has completed. The places we are seeing this aren't places where we're interacting with Expo directly. I've tried creating a project based on the Detox Repro Template that has Expo and reanimated in it but I haven't managed to get it to crash yet. Something I have noticed is that the amount of time it takes Detox to "react" (i.e. continue to the next command) to an animation finishing seems to vary somewhat - see the video below at 19 seconds vs the other repetitions from my attempted reproduction: test.movI also wondered if it might be a race between an animation finishing and a request completing, although again haven't managed to coax the reproduction into crashing. |
Are you using the template in CI? Do you have your code up somewhere so i can pull it or add to it? We only get these crashes in CI fyi |
^ that is a point. I don't think I've ever seen it locally. Fighting some other fires, but I'll get the template pushed somewhere when I can. |
|
@asafkorem I wonder if we can add reanimated, gesture handler, expo and react navigation to the official Detox e2e tests? By adding all of the common dependencies it might help to catch some of this crashing. What CI provider do you guys use for the project? |
Buildkite + Macstadium |
|
@shamilovtim sorry for the delay. https://github.com/PlayerData/DetoxReanimatedRepro Just trying to set up GitHub actions to see if that might result in a reproduction. |
|
@shamilovtim can I ask, do you run multiple simulators at once in CI (as in, do you use the |
|
Running PlayerData/DetoxReanimatedRepro#4 locally, Detox is consistently not waiting for animations to finish for some workers (between 1 and 3). It doesn't, however, Signal 11 |
|
To brain dump a bit more - the stack trace takes us to here (specifically, the block): Could Sorry to ping @LeoNatan, but I'm waaay out of my depth here - would you have any ideas around this space? |
No we don't. CircleCI doesn't allow hardware acceleration so we are limited performance wise |
I wonder if this has something to do with software rendered, slow CI VMs? Maybe the VM is so slow that |
|
Our CI doesn't use VMs but instead a pair of slightly ageing Mac Minis (2018) so the same theory holds. |
|
Couple of patches I tried yesterday. Firstly, this change which guards against the returned diff --git a/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Spies/CADisplayLink+DTXSpy.m b/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Spies/CADisplayLink+DTXSpy.m
index 023eabc..c296644 100644
--- a/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Spies/CADisplayLink+DTXSpy.m
+++ b/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Spies/CADisplayLink+DTXSpy.m
@@ -102,13 +102,16 @@ extern atomic_cfrunloop __RNRunLoop;
if(self.isPaused != paused)
{
id<DTXTimerProxy> proxy = [DTXTimerSyncResource existingTimerProxyWithDisplayLink:self create:NO];
- if(paused == YES)
- {
- [proxy untrack];
- }
- else
- {
- [self _detox_sync_trackIfNeeded];
+
+ if(proxy) {
+ if(paused == YES)
+ {
+ [proxy untrack];
+ }
+ else
+ {
+ [self _detox_sync_trackIfNeeded];
+ }
}
}This didn't help, which kinda makes sense - there should always be a proxy at this point if I've understood properly. The more interesting change is this one: diff --git a/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Spies/NSTimer+DTXSpy.m b/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Spies/NSTimer+DTXSpy.m
index 8f5549c..9a82b9e 100644
--- a/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Spies/NSTimer+DTXSpy.m
+++ b/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Spies/NSTimer+DTXSpy.m
@@ -78,7 +78,7 @@ CFRunLoopTimerRef __detox_sync_CFRunLoopTimerCreateWithHandler(CFAllocatorRef al
static void (*__orig_CFRunLoopAddTimer)(CFRunLoopRef rl, CFRunLoopTimerRef timer, CFRunLoopMode mode);
void __detox_sync_CFRunLoopAddTimer(CFRunLoopRef rl, CFRunLoopTimerRef timer, CFRunLoopMode mode)
{
-// NSLog(@"🤦♂️ addTimer: %@", NS(timer));
+ NSLog(@"🤦♂️ addTimer: %@", NS(timer));
id<DTXTimerProxy> trampoline = [DTXTimerSyncResource existingTimerProxyWithTimer:NS(timer)];
trampoline.runLoop = rl;
@@ -91,7 +91,7 @@ void __detox_sync_CFRunLoopAddTimer(CFRunLoopRef rl, CFRunLoopTimerRef timer, CF
static void (*__orig_CFRunLoopRemoveTimer)(CFRunLoopRef rl, CFRunLoopTimerRef timer, CFRunLoopMode mode);
void __detox_sync_CFRunLoopRemoveTimer(CFRunLoopRef rl, CFRunLoopTimerRef timer, CFRunLoopMode mode)
{
-// NSLog(@"🤦♂️ removeTimer: %@", NS(timer));
+ NSLog(@"🤦♂️ removeTimer: %@", NS(timer));
id<DTXTimerProxy> trampoline = [DTXTimerSyncResource existingTimerProxyWithTimer:NS(timer)];
[trampoline untrack];
@@ -102,7 +102,7 @@ void __detox_sync_CFRunLoopRemoveTimer(CFRunLoopRef rl, CFRunLoopTimerRef timer,
static void (*__orig_CFRunLoopTimerInvalidate)(CFRunLoopTimerRef timer);
void __detox_sync_CFRunLoopTimerInvalidate(CFRunLoopTimerRef timer)
{
-// NSLog(@"🤦♂️ invalidate: %@", NS(timer));
+ NSLog(@"🤦♂️ invalidate: %@", NS(timer));
id<DTXTimerProxy> trampoline = [DTXTimerSyncResource existingTimerProxyWithTimer:NS(timer)];
[trampoline untrack];
@@ -113,7 +113,7 @@ void __detox_sync_CFRunLoopTimerInvalidate(CFRunLoopTimerRef timer)
static void (*__orig___NSCFTimer_invalidate)(NSTimer* timer);
void __detox_sync___NSCFTimer_invalidate(NSTimer* timer)
{
- // NSLog(@"🤦♂️ invalidate: %@", timer);
+ NSLog(@"🤦♂️ invalidate: %@", timer);
id<DTXTimerProxy> trampoline = [DTXTimerSyncResource existingTimerProxyWithTimer:timer];
[trampoline untrack];
diff --git a/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Utils/_DTXTimerTrampoline.m b/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Utils/_DTXTimerTrampoline.m
index 5235fdc..b66d945 100644
--- a/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Utils/_DTXTimerTrampoline.m
+++ b/node_modules/detox/DetoxSync/DetoxSync/DetoxSync/Utils/_DTXTimerTrampoline.m
@@ -162,6 +162,8 @@ const void* __DTXTimerTrampolineKey = &__DTXTimerTrampolineKey;
return;
}
+ NSLog(@"🤦♂️ track: %@", _timer);
+
_tracking = YES;
[DTXTimerSyncResource.sharedInstance trackTimerTrampoline:self];
}
@@ -173,7 +175,7 @@ const void* __DTXTimerTrampolineKey = &__DTXTimerTrampolineKey;
return;
}
- // NSLog(@"🤦♂️ untrack: %@", _timer);
+ NSLog(@"🤦♂️ untrack: %@", _timer);
[DTXTimerSyncResource.sharedInstance untrackTimerTrampoline:self];
_tracking = NO;All this change is doing is re-enabling some of the commented out logging (and adding a little bit extra). With this change, I wasn't able to reproduce a segfault over around 10 runs (usually, I would have seen at least one if not two in that space). Presumably, adding the logging is causing a reference to the timer to be retained for longer than it would be otherwise. Either that, or the extra slowness of logging makes the race no longer happen. |
|
valid |
|
I once repro'd this locally on my M1 Max . Wonder how many Detox runs it would take to see it again. |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
@matthewparavati if you can reliably reproduce your issue, that sounds like a different problem. I'd suggest creating a reproduction repo based on https://github.com/wix-incubator/DetoxTemplate and filing a separate issue. |
|
Signal 6 is different than Signal 11. In most cases we have found that Signal 6 is a valid crash in your business logic and a problem in your app. |
|
Thanks for the responses @ball-hayden and @shamilovtim. Apparently our android e2e tests were a false positive pass. They were actually crashing, too, but for some reason said they finished as passing 😕 I'll look to create a new issue |
|
@shamilovtim In the next reproduction of this issue... Could you be kind enough to provide all of the up-to-date artifacts and info so we could take another round at investigating this? Or better yet, let's take this up on Discord. |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions! For more information on bots in this repository, read this discussion. |
|
Still seeing this in the latest versions. @d4vidi what artifacts and further information would you like - I can make sure our CI is archiving them so I can send them over? |
|
The issue has been closed for inactivity. |
|
Oh. Okay. Definitely still seeing this. Sorry. |
Screenshots, videos, logs, view-hierarchy - Essentially everything 😄 You can start here |
|
+1 seeing this in latest versions. Will try and gather some logs if possible |
|
also getting the 1. most frequent:Signal 11 was raised ( 0 Detox 0x0000000102727118 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 36 1 Detox 0x0000000102729d28 __DTXHandleCrash + 440 2 Detox 0x000000010272a358 __DTXHandleSignal + 72 3 libsystem_platform.dylib 0x00000001b0591760 _sigtramp + 52 4 STAGING 0x0000000100ae51f4 reanimated::Scheduler::triggerUI() + 92 5 STAGING 0x0000000100ae51f4 reanimated::Scheduler::triggerUI() + 92 6 DetoxSync 0x000000010d5a0454 ____detox_sync_dispatch_wrapper_block_invoke + 44 7 libdispatch.dylib 0x0000000180133fa4 _dispatch_call_block_and_release + 24 8 libdispatch.dylib 0x0000000180135768 _dispatch_client_callout + 16 9 libdispatch.dylib 0x0000000180145018 _dispatch_main_queue_drain + 1220 10 libdispatch.dylib 0x0000000180144b44 _dispatch_main_queue_callback_4CF + 40 11 CoreFoundation 0x0000000180372ca4 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 12 12 CoreFoundation 0x000000018036d360 __CFRunLoopRun + 1956 13 CoreFoundation 0x000000018036c7a4 CFRunLoopRunSpecific + 584 14 GraphicsServices 0x0000000188ff7c98 GSEventRunModal + 160 15 UIKitCore 0x000000011021237c -[UIApplication _run] + 868 16 DetoxSync 0x000000010d5a68fc __detox_sync_UIApplication_run + 376 17 UIKitCore 0x0000000110216374 UIApplicationMain + 124 18 STAGING 0x00000001008dabdc main + 80 19 dyld 0x000000010235dfa0 start_sim + 20 20 ??? 0x00000001022ad08c 0x0 + 4331327628 21 ??? 0xce6a000000000000 0x0 + -3573043354365067264 ) 2. once every 10 random crashes approximately:Signal 11 was raised ( 0 Detox 0x00000001028bb118 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 36 1 Detox 0x00000001028bdd28 __DTXHandleCrash + 440 2 Detox 0x00000001028be358 __DTXHandleSignal + 72 3 libsystem_platform.dylib 0x00000001b0591760 _sigtramp + 52 4 STAGING 0x0000000100dc11f4 reanimated::Scheduler::triggerUI() + 92 5 STAGING 0x0000000100dc11f4 reanimated::Scheduler::triggerUI() + 92 6 STAGING 0x0000000100d9c49c invocation function for block in reanimated::createReanimatedModule(RCTBridge*, std::__1::shared_ptr) + 60 7 STAGING 0x0000000100daae54 -[REAAnimationsManager onViewCreate:after:] + 84 8 STAGING 0x0000000100dbb450 __51-[REAUIManager uiBlockWithLayoutUpdateForRootView:]_block_invoke.45 + 1824 9 STAGING 0x0000000100e57c50 __44-[RCTUIManager flushUIBlocksWithCompletion:]_block_invoke + 164 10 STAGING 0x0000000100e57d40 __44-[RCTUIManager flushUIBlocksWithCompletion:]_block_invoke.146 + 28 11 DetoxSync 0x0000000114c5c454 ____detox_sync_dispatch_wrapper_block_invoke + 44 12 libdispatch.dylib 0x0000000180133fa4 _dispatch_call_block_and_release + 24 13 libdispatch.dylib 0x0000000180135768 _dispatch_client_callout + 16 14 libdispatch.dylib 0x0000000180145018 _dispatch_main_queue_drain + 1220 15 libdispatch.dylib 0x0000000180144b44 _dispatch_main_queue_callback_4CF + 40 16 CoreFoundation 0x0000000180372ca4 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 12 17 CoreFoundation 0x000000018036d360 __CFRunLoopRun + 1956 18 CoreFoundation 0x000000018036c7a4 CFRunLoopRunSpecific + 584 19 GraphicsServices 0x0000000188ff7c98 GSEventRunModal + 160 20 UIKitCore 0x0000000103fe637c -[UIApplication _run] + 868 21 DetoxSync 0x0000000114c628fc __detox_sync_UIApplication_run + 376 22 UIKitCore 0x0000000103fea374 UIApplicationMain + 124 23 STAGING 0x0000000100bb6bdc main + 80 24 dyld 0x00000001023edfa0 start_sim + 20 25 ??? 0x000000010249108c 0x0 + 4333310092 26 ??? 0x4a72000000000000 0x0 + 5364350106151682048 ) as you can see, in our case, the crash seems to be related to reanimated (firebase or expo aren't installed/used). update: also seeing crashes on android (here again, reanimated seems to be involved):
|
Yeah - that's about as far we've narrowed it down to. Our workaround is to stub out everything that's animated with a non-animated variant, which isn't great :-( |
|
For me this is related to Edit: I mocked all files, that import suspicious modules and so far both |
|
@d4vidi @asafkorem we might need to set up a test suite with popular libraries someday. 😌 |
|
^ ultimately, the problems here come down to Detox's tracking of animations. Any library that triggers animations ( If we can solve the underlying issue in the handling of animations through the trampolines, this problem will go away (and we probably don't need to have a test suite with popular libraries - a basic test of animation tracking functionality should be sufficient). |
|
@ball-hayden Detox has some basic self-test suite for animations: https://github.com/wix/Detox/blob/master/detox/test/e2e/12.animations.test.js Seems like it is not sufficient. A good question would be "what's missing". Here's the source code of the Animations screen: https://github.com/wix/Detox/blob/master/detox/test/src/Screens/AnimationsScreen.js |
|
Try animations that use JSI. I've experienced many JSI crashes due to reload, hard reload or hot reload. So reanimated. |
|
I had this error for over a year now. Finally solved it by upgrading |
Description
A clear and concise description of what the bug is.
Reproduction
We still see Signal 11 crashes in our project even after adding both
SIMCTL_CHILD_NSZombieEnabled=YES detox test -c ios.sim.releaseas well as updating firebase + firebase perf to the latest versions and ensuring that GoogleUtilities are at 7.7+.The error is commonly reproduced on CircleCI and is incredibly difficult / practically impossible to reproduce locally.
In order to help debug I've provided a list of all of the dependencies in our project. We use Expo 43 (no expo updates), React Native 0.65.2 and Codepush (probably unrelated, the error happens in the middle of a test, not at the beginning). I've found that the Stripe iOS SDK (via tipsi-stripe) is a source of method swizzling. Also, while Flipper is a source of method swizzling, we are building a release project so Flipper should be excluded.
Let me know if there's anything else I can provide to help debug and resolve this issue.
The stacktrace is:
List of dependencies present in the project:
Expected behavior
Expect no signal 11
Screenshots / Video
Not necessary right now, will add later if needed.
Environment
jest-circusLogs
Device and verbose Detox logs
--loglevel traceargument and am providing the verbose log below:Detox logs
The stacktrace is:The text was updated successfully, but these errors were encountered: