Crash in DetoxSync #2641

denissb · 2021-02-15T09:15:59Z

I have an issue when running tests in parallel with --workers 2 using jest-circus after upgrading to detox 18
The form tests that use the keyboard actions like typeText and clearText fail when they are running in parallel with another test.

Reproduction

Have 2 tests, one that is verifying some on screen behaviour (fx. navigation or button actions) and one that is testing some form actions like entering and removing text in inputs to check form validation logic
Run the tests in parallel with --workers 2 using jest-circus as a runner
See the tests fail with random errors and a sync issue

I also get this error thrown in one of the tests when running in parallel

thrown: "Signal 11 was raised
    (
    	0   Detox                               0x000000010aa97ef5 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 37
    	1   Detox                               0x000000010aa9b100 __DTXHandleCrash + 464
    	2   Detox                               0x000000010aa9b845 __DTXHandleSignal + 59
    	3   libsystem_platform.dylib            0x00007fff6115a5fd _sigtramp + 29
    	4   ???                                 0x0000000000000000 0x0 + 0
    	5   DetoxSync                           0x000000010b861aaf -[_DTXTimerTrampoline fire:] + 188
    	6   DetoxSync                           0x000000010b849ef0 _DTXCFTimerTrampoline + 74
    	7   CoreFoundation                      0x00007fff20390c57 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 20
    	8   CoreFoundation                      0x00007fff2039072a __CFRunLoopDoTimer + 926
    	9   CoreFoundation                      0x00007fff2038fcdd __CFRunLoopDoTimers + 265
    	10  CoreFoundation                      0x00007fff2038a35e __CFRunLoopRun + 1949
    	11  CoreFoundation                      0x00007fff203896d6 CFRunLoopRunSpecific + 567
    	12  App                          0x00000001080f2fd1 +[RCTCxxBridge runRunLoop] + 993
    	13  DetoxSync                           0x000000010b8523c0 swz_runRunLoopThread + 291
    	14  Foundation                          0x00007fff20858e68 __NSThread__start__ + 1042
    	15  libsystem_pthread.dylib             0x00007fff61167109 _pthread_start + 148
    	16  libsystem_pthread.dylib             0x00007fff61162b8b thread_start + 15
    )"

Expected behaviour

The tests should pass just like when you run them with --maxWorkers 1

Environment (please complete the following information):

Detox: 18.3.1
React Native: 0.63.4
Node: 14.15.4
Device: iPhone 8
Xcode: 12.3
iOS: 14.4
macOS: 10.15.7
I have followed the instructions under Identifying which synchronization mechanism causes us to wait too much, I have read synchronization debug documentation and am providing the relevant synchronization debug output below:

The text was updated successfully, but these errors were encountered:

LeoNatan · 2021-02-15T15:21:00Z

This looks like a crash in DetoxSync. @alon-ha @d4vidi

    	4   ???                                 0x0000000000000000 0x0 + 0
    	5   DetoxSync                           0x000000010b861aaf -[_DTXTimerTrampoline fire:] + 188
    	6   DetoxSync                           0x000000010b849ef0 _DTXCFTimerTrampoline + 74

It could be a bad app code, but it could be a bug in DetoxSync.

LeoNatan · 2021-02-15T15:22:03Z

I don't think the number of workers matters here.

LeoNatan · 2021-02-15T15:28:14Z

Might be similar to #2627 (which was closed in error as duplicate of another issue).

denissb · 2021-02-15T16:01:41Z

I don't think the number of workers matters here.

Maybe it doesn't but when I run the tests without --workers 2 they are rock solid and pass constantly. I also didn't face any issues with the same test suit running on Detox 17.14.8 whatsoever.

LeoNatan · 2021-02-15T16:54:13Z

Detox 18 has a new synchronization mechanism, built in-house.

denissb · 2021-02-15T17:57:17Z

Yes, but it is quite nice. Thats why I want to upgrade, so now I am also digging around trying to find the cause.
I will try to upgrade to 18.6.0 and see if it changes anything.

sam-barker · 2021-02-17T09:00:54Z

@denissb Please comment if you find the same crash when upgrading to the latest v18. When I last tried it happened on 18.3

d4vidi · 2021-02-17T10:44:13Z

@alon-ha any leads to help out?

denissb · 2021-02-17T12:47:31Z

@sam-barker I still have the same issue on 18.6.0

sam-barker · 2021-03-01T10:57:34Z

Any movement on this at all?

DanielMarkiel · 2021-03-01T12:08:36Z

Hi,
I'm facing the same issue on my project but with --maxWorkers 1.
All tests are passing fine on Android emulator.
Tests were passing fine on iOS with Detox: 17.14.9.

Environment:

Detox: 18.6.2
jest-circus: 26.6.3
React Native: 0.63.4
Node: 14.16.0
Device: iPhone 8
Xcode: 12.4
iOS: 14.4
macOS: 11.2.1

Logs:

thrown: "Signal 11 was raised
    (
    	0   Detox                               0x000000010fec6eb5 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 37
    	1   Detox                               0x000000010feca0c0 __DTXHandleCrash + 464
    	2   Detox                               0x000000010feca805 __DTXHandleSignal + 59
    	3   libsystem_platform.dylib            0x00007fff6115cd7d _sigtramp + 29
    	4   libsystem_c.dylib                   0x00007fff86d4a728 __global_locale + 0
    	5   CoreFoundation                      0x00007fff20490049 -[__NSDictionaryM dealloc] + 128
    	6   libobjc.A.dylib                     0x00007fff2018f834 objc_object::sidetable_release(bool, bool) + 174
    	7   libobjc.A.dylib                     0x00007fff2018bb85 _object_remove_assocations + 580
    	8   libobjc.A.dylib                     0x00007fff20188f25 objc_destructInstance + 84
    	9   libobjc.A.dylib                     0x00007fff2018efb8 -[NSObject dealloc] + 21
    	10  App                       0x000000010eac59a5 -[FBSDKGraphRequestConnection dealloc] + 96
    	11  libobjc.A.dylib                     0x00007fff2018f834 objc_object::sidetable_release(bool, bool) + 174
    	12  CFNetwork                           0x00007fff236d0b10 _CFNetworkHTTPConnectionCacheSetLimit + 163652
    	13  CFNetwork                           0x00007fff23519ec5 CFNetwork + 28357
    	14  DetoxSync                           0x0000000111d1b270 ____detox_sync_dispatch_wrapper_block_invoke + 23
    	15  libdispatch.dylib                   0x00007fff2010532f _dispatch_call_block_and_release + 12
    	16  libdispatch.dylib                   0x00007fff20106508 _dispatch_client_callout + 8
    	17  libdispatch.dylib                   0x00007fff2010c3f9 _dispatch_lane_serial_drain + 715
    	18  libdispatch.dylib                   0x00007fff2010cfab _dispatch_lane_invoke + 458
    	19  libdispatch.dylib                   0x00007fff20117577 _dispatch_workloop_worker_thread + 782
    	20  libsystem_pthread.dylib             0x00007fff61167499 _pthread_wqthread + 314
    	21  libsystem_pthread.dylib             0x00007fff61166467 start_wqthread + 15
    )"

   4 |
      5 | describe('Intro', () => {
    > 6 |   beforeAll(async () => {
        |   ^
      7 |     await device.launchApp(deviceLaunchAppConfig);
      8 |     await device.setURLBlacklist(blacklistedURLs);
      9 |   });

stale · 2021-03-31T17:28:46Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this reporsitory, read this discussion.

d4vidi · 2021-03-31T19:21:23Z

All, we are currently on a break and a bit short-staffed at the moment, in general. We hope to get on top of this within 1-2 weeks. Here are similar discussions that could be relevant, in the meantime:

shamilovtim · 2021-04-13T13:58:41Z

Hello, any updates on this? We are sorely in need of a Firebase upgrade (the old version of Firebase is actually causing bugs in our app) but we can't bump Firebase until this gets resolved.

wdospinal · 2021-04-13T19:12:43Z

Hi, any advice on this, the app is having this issue when trying to add firebase and segment to the project

noomorph · 2021-04-14T04:30:31Z

@alon-ha, pinging you just for visibility.

Personally, I think this is an important issue since many RN users are using Firebase including our teams as well if I am not mistaken.

Let's check if we already have clear instructions on how to reproduce the crash. @wdospinal, @shamilovtim could you help to locate any demo project crashing with that error or create one?

I think Alon will get back to his job duties starting from the next week, and I hope he'll find time during April-May to check out a demo project if it will be already prepared.

shamilovtim · 2021-04-14T04:39:10Z

From what I could tell based on shallow research the issue might be caused by monkey patching in HTTP libraries. That's supposedly how it appeared in KTOR: firebase/firebase-ios-sdk#7277

I've seen other tickets in the ecosystem also blaming the same thing in various ways

shamilovtim · 2021-04-14T04:41:31Z

Also related: JetBrains/kotlin-native#4569 (blame was again monkey patching)

noomorph · 2021-04-14T04:43:19Z

@shamilovtim, what but isn't that Android-related there? At the moment, we are discussing iOS.

This workaround solves the issue described here: firebase/firebase-ios-sdk#9083 And some of the crashes that was mentioned here: - wix#3000 - wix#3123 - wix#2641 - wix#2802

This workaround solves the issue described here: firebase/firebase-ios-sdk#9083 And some of the crashes that was mentioned here: - #3000 - #3123 - #2641 - #2802

asafkorem · 2021-12-22T09:33:04Z

#3135 was merged, it should solve the issue of DetoxSync crashes when Firebase/Performance is integrated, it was tested on the example apps that reproduced this issue.
@tomphilbin @shamilovtim @wdospinal CC

Once you have updated a version (to the latest/next version), if any of you are still having Signal 11 crashes on DetoxSync when Firebase/Performance is integrated, please open a new bug report with the required details.

As for the parallel --workers issue, I think that there's still missing information here, since I fail to reproduce it myself @denissb
Can any of the reporters reproduce this issue with our template app? https://github.com/wix-incubator/DetoxTemplate

@param-finder I will investigate soon the crashes with Expo.
Can you please open a designated issue for this bug you have?

ball-hayden · 2022-01-05T15:49:42Z

Sorry to be the bearer of bad news, but we're still seeing Signal 11's with Detox 19.4.1:

Signal 11 was raised
(
	0   Detox                               0x000000011217b6a5 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 37
	1   Detox                               0x000000011217e230 __DTXHandleCrash + 464
	2   Detox                               0x000000011217e971 __DTXHandleSignal + 59
	3   libsystem_platform.dylib            0x00007fff60335d7d _sigtramp + 29
	4   ???                                 0x0000000000000125 0x0 + 293
	5   DetoxSync                           0x0000000113f27b37 -[_DTXTimerTrampoline fire:] + 188
	6   DetoxSync                           0x0000000113f0fb70 _DTXCFTimerTrampoline + 74
	7   CoreFoundation                      0x00007fff2039178a __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 20
	8   CoreFoundation                      0x00007fff2039127c __CFRunLoopDoTimer + 924
	9   CoreFoundation                      0x00007fff2039081a __CFRunLoopDoTimers + 265
	10  CoreFoundation                      0x00007fff2038ae69 __CFRunLoopRun + 2013
	11  CoreFoundation                      0x00007fff2038a1a7 CFRunLoopRunSpecific + 567
	12  mobile                              0x000000010d6e34e2 +[RCTCxxBridge runRunLoop] + 962
	13  DetoxSync                           0x0000000113f1773a swz_runRunLoopThread + 291
	14  Foundation                          0x00007fff2085c8a9 __NSThread__start__ + 1068
	15  libsystem_pthread.dylib             0x00007fff603438fc _pthread_start + 224
	16  libsystem_pthread.dylib             0x00007fff6033f443 thread_start + 15
)

It looks similar to the original trace on this issue, but happy to open a new one if you'd prefer @asafkorem ?

asafkorem · 2022-01-05T15:53:16Z

Hi @ball-hayden, thanks for the report. Are you running parallel tests?

ball-hayden · 2022-01-06T19:59:26Z

We're running with --workers 2 --keepLockFile, yes.

sergiomap · 2022-01-06T22:23:12Z

we also still have the Signal 11 error running tests in parallel.

s-aleinik · 2022-01-21T10:20:52Z

I've also got interesting behaviour (no parallel workers used):

..detox[73724] ERROR: [WS_ERROR] The app has crashed, see the details below:

Signal 11 was raised
(
        0   Detox                               0x000000010a4556a5 +[NSThread(DetoxUtils) dtx_demangledCallStackSymbols] + 37
        1   Detox                               0x000000010a458230 __DTXHandleCrash + 464
        2   Detox                               0x000000010a458971 __DTXHandleSignal + 59
        3   libsystem_platform.dylib            0x00007fff6da24e2d _sigtramp + 29
        4   ???                                 0x0000600001050000 0x0 + 105553133371392
        5   CoreFoundation                      0x00007fff2046fd0b -[__NSDictionaryM dealloc] + 128
        6   libobjc.A.dylib                     0x00007fff2019f9f7 objc_object::sidetable_release(bool, bool) + 177
        7   libobjc.A.dylib                     0x00007fff2019bcb3 _object_remove_assocations + 578
        8   libobjc.A.dylib                     0x00007fff20199207 objc_destructInstance + 84
        9   libobjc.A.dylib                     0x00007fff2019f187 -[NSObject dealloc] + 21
        10  libobjc.A.dylib                     0x00007fff2019f9f7 objc_object::sidetable_release(bool, bool) + 177
        11  CFNetwork                           0x00007fff23e28704 _CFHTTPMessageSetResponseProxyURL + 12873
        12  libsystem_blocks.dylib              0x00007fff2008b434 _Block_release + 130
        13  Foundation                          0x00007fff207e2180 -[NSBlockOperation dealloc] + 64
        14  libobjc.A.dylib                     0x00007fff2019f9f7 objc_object::sidetable_release(bool, bool) + 177
        15  libobjc.A.dylib                     0x00007fff201a1175 AutoreleasePoolPage::releaseUntil(objc_object**) + 175
        16  libobjc.A.dylib                     0x00007fff201a1064 objc_autoreleasePoolPop + 185
        17  CFNetwork                           0x00007fff24060e96 __CFTubeSetTubeTypeNotifier + 106626
        18  CFNetwork                           0x00007fff24060d95 __CFTubeSetTubeTypeNotifier + 106369
        19  CFNetwork                           0x00007fff23f8bfd5 _CFNetworkSetATSContext + 52017
        20  CFNetwork                           0x00007fff23f8e36a _CFNetworkSetATSContext + 61126
        21  CFNetwork                           0x00007fff23f8e2da _CFNetworkSetATSContext + 60982
        22  CFNetwork                           0x00007fff23f886a5 _CFNetworkSetATSContext + 37377
        23  DetoxSync                           0x000000010a253b27 ____detox_sync_dispatch_wrapper_block_invoke + 23
        24  libdispatch.dylib                   0x00007fff2011265a _dispatch_call_block_and_release + 12
        25  libdispatch.dylib                   0x00007fff2011383a _dispatch_client_callout + 8
        26  libdispatch.dylib                   0x00007fff20119e0c _dispatch_lane_serial_drain + 743
        27  libdispatch.dylib                   0x00007fff2011aa07 _dispatch_lane_invoke + 455
        28  libdispatch.dylib                   0x00007fff2012557d _dispatch_workloop_worker_thread + 772
        29  libsystem_pthread.dylib             0x00007fff6da2f074 _pthread_wqthread + 326
        30  libsystem_pthread.dylib             0x00007fff6da2dffb start_wqthread + 15
)

There is Signal 11 error. But all my tests are running successfully after it.

RN: 0.66.2
Detox 19.4.2

Firebase packages:

@react-native-firebase/analytics: 12.0.0
@react-native-firebase/app: 12.0.0
@react-native-firebase/crashlytics: 12.0.0
@react-native-firebase/dynamic-links: 12.0.0
@react-native-firebase/messaging: 12.0.0
@react-native-firebase/perf: 12.0.0
@react-native-firebase/remote-config: 12.0.0

s-aleinik · 2022-01-24T09:50:18Z

After updating all the above @react-native-firebase packages to 14.2.4 issue was resolved for me

stale · 2022-02-23T22:14:35Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this reporsitory, read this discussion.

ball-hayden · 2022-02-24T09:24:29Z

We're still seeing Signal 11s intermittently.

(I appreciate this isn't massively helpful - I'll keep trying to narrow it down)

stale · 2022-03-26T21:01:32Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this reporsitory, read this discussion.

ball-hayden · 2022-03-28T07:56:25Z

Still seeing this.
I still haven't been able to narrow it down though.

d4vidi · 2022-03-28T14:39:10Z

@ball-hayden would appreciate more info indeed. Thank you.

shamilovtim · 2022-03-28T14:53:15Z

Can we move to this thread? #3207

ball-hayden · 2022-03-28T16:00:00Z

Yeah - @d4vidi there's some more information in #3207.

Both myself and @shamilovtim are struggling to isolate what causes this, so I appreciate there isn't a huge amount that can be done at the moment.

It is still an issue, however, so IMHO shouldn't be marked as stale.

stale · 2022-05-30T21:11:37Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this reporsitory, read this discussion.

d4vidi · 2022-05-31T09:14:03Z

Closing in favor of #3207

denissb added type: triage/bug platform: ios labels Feb 15, 2021

denissb changed the title ~~[v18] Tests failing randomly when running in parallel on xcode 12.3~~ [v18] Tests failing randomly when running in parallel Feb 15, 2021

LeoNatan changed the title ~~[v18] Tests failing randomly when running in parallel~~ Crash in DetoxSync Feb 15, 2021

LeoNatan added type: bug 🐞 and removed type: triage/bug labels Feb 15, 2021

LeoNatan assigned alon-ha Feb 15, 2021

d4vidi added the z_domain: detox18 label Feb 24, 2021

deermichel mentioned this issue Mar 31, 2021

Signal 11 was raised #2706

Closed

1 task

stale bot added the 🏚 stale label Mar 31, 2021

stale bot removed the 🏚 stale label Mar 31, 2021

asafkorem mentioned this issue Dec 15, 2021

fix(iOS, AppleSimUtils): enable NSZombies on app launch. #3135

Merged

asafkorem added a commit that referenced this issue Dec 20, 2021

iOS(AppleSimUtils): enable NSZombies on app launch.

cf5249b

This workaround solves the issue described here: firebase/firebase-ios-sdk#9083 And some of the crashes that was mentioned here: - #3000 - #3123 - #2641 - #2802

asafkorem linked a pull request Dec 22, 2021 that will close this issue

fix(iOS, AppleSimUtils): enable NSZombies on app launch. #3135

Merged

asafkorem closed this as completed in #3135 Dec 22, 2021

asafkorem reopened this Dec 22, 2021

stale bot added the 🏚 stale label Feb 23, 2022

stale bot removed the 🏚 stale label Feb 24, 2022

stale bot added the 🏚 stale label Mar 26, 2022

stale bot removed the 🏚 stale label Mar 28, 2022

d4vidi added status: accepted and removed domain: detox18 labels Apr 13, 2022

stale bot added the 🏚 stale label May 30, 2022

d4vidi closed this as completed May 31, 2022

Crash in DetoxSync #2641

Crash in DetoxSync #2641

Comments

denissb commented Feb 15, 2021 • edited Loading

Reproduction

Expected behaviour

Environment (please complete the following information):

LeoNatan commented Feb 15, 2021

LeoNatan commented Feb 15, 2021

LeoNatan commented Feb 15, 2021

denissb commented Feb 15, 2021 • edited Loading

LeoNatan commented Feb 15, 2021

denissb commented Feb 15, 2021

sam-barker commented Feb 17, 2021

d4vidi commented Feb 17, 2021

denissb commented Feb 17, 2021

sam-barker commented Mar 1, 2021

DanielMarkiel commented Mar 1, 2021

stale bot commented Mar 31, 2021

d4vidi commented Mar 31, 2021

shamilovtim commented Apr 13, 2021

wdospinal commented Apr 13, 2021 • edited Loading

noomorph commented Apr 14, 2021

shamilovtim commented Apr 14, 2021

shamilovtim commented Apr 14, 2021

noomorph commented Apr 14, 2021 • edited Loading

asafkorem commented Dec 22, 2021 • edited Loading

ball-hayden commented Jan 5, 2022

asafkorem commented Jan 5, 2022 • edited Loading

ball-hayden commented Jan 6, 2022

sergiomap commented Jan 6, 2022

s-aleinik commented Jan 21, 2022 • edited Loading

s-aleinik commented Jan 24, 2022

stale bot commented Feb 23, 2022

ball-hayden commented Feb 24, 2022 • edited Loading

stale bot commented Mar 26, 2022

ball-hayden commented Mar 28, 2022

d4vidi commented Mar 28, 2022

shamilovtim commented Mar 28, 2022

ball-hayden commented Mar 28, 2022

stale bot commented May 30, 2022

d4vidi commented May 31, 2022

denissb commented Feb 15, 2021 •

edited

Loading

denissb commented Feb 15, 2021 •

edited

Loading

wdospinal commented Apr 13, 2021 •

edited

Loading

noomorph commented Apr 14, 2021 •

edited

Loading

asafkorem commented Dec 22, 2021 •

edited

Loading

asafkorem commented Jan 5, 2022 •

edited

Loading

s-aleinik commented Jan 21, 2022 •

edited

Loading

ball-hayden commented Feb 24, 2022 •

edited

Loading