Test does not start on iOS during patrol develop until Hot Restart is triggered

[prasannavenkateshpalanisamy]

Steps to reproduce

1. Do the setup as mentioned in the patrol official website.
2. Run using patrol develop command with your project.

Actual results

Once after running the patrol develop -t integration_test/example_test.dart, the application is launching without TEST STARTING screen but it will appear once after hot restart clicked after it enabled and the testing will carry on.

Logs

MacBook-Pro Next-Gen-App % patrol develop -t integration_test/example_test.dart --verbose
In next major release, patrolTest method will be intended for UI tests.
If you want to use Patrol in your widget tests, use patrol_finders package.

For more information, see https://patrol.leancode.co/patrol-finders-release
Disable this warning by setting the PATROL_FINDERS environment variable.
      
Verbose mode enabled. More logs will be printed.
Received test target: /Users/lallu/Documents/GitHub/Next-Gen-App/integration_test/example_test.dart
Generated entrypoint /Users/lallu/Documents/GitHub/Next-Gen-App/integration_test/test_bundle.dart for development
Received Android flavor: jk
Received iOS flavor: jk
$ flutter --no-version-check devices --machine
No device specified, using the first one (iPhone 14 Plus)
Received device: iPhone 14 Plus
Received 8 --dart-define(s) (0 custom, 8 internal)
Received internal --dart-define: PATROL_WAIT=0
Received internal --dart-define: PATROL_APP_PACKAGE_NAME=com.paytronix.nexgenfl
Received internal --dart-define: PATROL_APP_BUNDLE_ID=com.paytronix.nexgenfl
Received internal --dart-define: PATROL_ANDROID_APP_NAME=NGA JK Cafe
Received internal --dart-define: PATROL_IOS_APP_NAME=NGA JK Cafe
Received internal --dart-define: INTEGRATION_TEST_SHOULD_REPORT_RESULTS_TO_NATIVE=false
Received internal --dart-define: PATROL_TEST_LABEL_ENABLED=true
Received internal --dart-define: PATROL_HOT_RESTART=true
• Building app with entrypoint test_bundle.dart for iOS simulator (debug)...
$ flutter build ios --no-version-check --config-only --no-codesign --debug --simulator --flavor jk --target /Users/lallu/Documents/GitHub/Next-Gen-App/integration_test/test_bundle.dart --dart-define PATROL_WAIT=0 --dart-define PATROL_APP_PACKAGE_NAME=com.paytronix.nexgenfl --dart-define PATROL_APP_BUNDLE_ID=com.paytronix.nexgenfl --dart-define PATROL_ANDROID_APP_NAME=NGA JK Cafe --dart-define PATROL_IOS_APP_NAME=NGA JK Cafe --dart-define INTEGRATION_TEST_SHOULD_REPORT_RESULTS_TO_NATIVE=false --dart-define PATROL_TEST_LABEL_ENABLED=true --dart-define PATROL_HOT_RESTART=true
        Building com.paytronix.nexgenfl for simulator (ios)...
$ xcodebuild build-for-testing -workspace Runner.xcworkspace -scheme jk -configuration Debug-jk -only-testing RunnerUITests -sdk iphonesimulator -destination generic/platform=iOS Simulator -quiet -derivedDataPath ../build/ios_integ OTHER_SWIFT_FLAGS=$(inherited) -D PATROL_ENABLED
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Runner/iOSDatePicker.swift:56:41: warning: 'keyWindow' was deprecated in iOS 13.0: Should not be used for applications that support multiple scenes as it returns a key window across all connected scenes
                let win  = UIApplication.shared.keyWindow
                                                ^
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Runner/iOSPicker.swift:56:41: warning: 'keyWindow' was deprecated in iOS 13.0: Should not be used for applications that support multiple scenes as it returns a key window across all connected scenes
                let win  = UIApplication.shared.keyWindow
                                                ^
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Runner/iOSDatePicker.swift:56:41: warning: 'keyWindow' was deprecated in iOS 13.0: Should not be used for applications that support multiple scenes as it returns a key window across all connected scenes
                let win  = UIApplication.shared.keyWindow
                                                ^
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Runner/iOSPicker.swift:56:41: warning: 'keyWindow' was deprecated in iOS 13.0: Should not be used for applications that support multiple scenes as it returns a key window across all connected scenes
                let win  = UIApplication.shared.keyWindow
                                                ^
        note: Building targets in dependency order
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 9.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'CocoaAsyncSocket' from project 'Pods')
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 9.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'PromisesObjC' from project 'Pods')
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 9.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'GoogleUtilities' from project 'Pods')
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 10.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'FirebaseCoreInternal' from project 'Pods')
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 10.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'FirebaseCore' from project 'Pods')
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 10.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'FirebaseInstallations' from project 'Pods')
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 9.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'nanopb' from project 'Pods')
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 9.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'GoogleDataTransport' from project 'Pods')
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 9.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'HTTPParserC' from project 'Pods')
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 9.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'ReachabilitySwift' from project 'Pods')
        /Users/lallu/Documents/GitHub/Next-Gen-App/ios/Pods/Pods.xcodeproj: warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 9.0, but the range of supported deployment target versions is 11.0 to 16.2.99. (in target 'Telegraph' from project 'Pods')
        note: Run script build phase 'Run Script' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'Runner' from project 'Runner')
        note: Run script build phase 'Thin Binary' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'Runner' from project 'Runner')
        warning: Run script build phase 'xcode_backend build' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'RunnerUITests' from project 'Runner')
        warning: Run script build phase 'xcode_backend embed_and_thin' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'RunnerUITests' from project 'Runner')
✓ Completed building app with entrypoint test_bundle.dart for iOS simulator (94.5s)
Will uninstall apps before running tests
$ xcrun simctl uninstall 34EBC3CC-F7EA-40F4-8730-EA5891362A5F com.paytronix.nexgenfl
$ xcrun simctl uninstall 34EBC3CC-F7EA-40F4-8730-EA5891362A5F com.paytronix.nexgenfl.RunnerUITests.xctrunner
• Running app with entrypoint test_bundle.dart for iOS simulator on simulator iPhone 14 Plus...
$ xcodebuild -showsdks -json
Interactive shell mode enabled.
Logs: waiting for them...
$ flutter --no-version-check logs --device-id 34EBC3CC-F7EA-40F4-8730-EA5891362A5F
$ flutter attach --no-version-check --debug --device-id 34EBC3CC-F7EA-40F4-8730-EA5891362A5F --app-id com.paytronix.nexgenfl --target /Users/lallu/Documents/GitHub/Next-Gen-App/integration_test/test_bundle.dart --dart-define PATROL_WAIT=0 --dart-define PATROL_APP_PACKAGE_NAME=com.paytronix.nexgenfl --dart-define PATROL_APP_BUNDLE_ID=com.paytronix.nexgenfl --dart-define PATROL_ANDROID_APP_NAME=NGA JK Cafe --dart-define PATROL_IOS_APP_NAME=NGA JK Cafe --dart-define INTEGRATION_TEST_SHOULD_REPORT_RESULTS_TO_NATIVE=false --dart-define PATROL_TEST_LABEL_ENABLED=true --dart-define PATROL_HOT_RESTART=true
Hot Restart: waiting for attach to the app...
Assuming SDK version 16.2 for iphonesimulator
Looking for .xctestrun matching jk_*iphonesimulator16.2*.xctestrun at /Users/lallu/Documents/GitHub/Next-Gen-App/build/ios_integ/Build/Products
Found 1 match(es), the first one will be used
Found /Users/lallu/Documents/GitHub/Next-Gen-App/build/ios_integ/Build/Products/jk_iphonesimulator16.2-arm64-x86_64.xctestrun
$ xcodebuild test-without-building -xctestrun /Users/lallu/Documents/GitHub/Next-Gen-App/build/ios_integ/Build/Products/jk_iphonesimulator16.2-arm64-x86_64.xctestrun -only-testing RunnerUITests -destination platform=iOS Simulator,name=iPhone 14 Plus -resultBundlePath /Users/lallu/Documents/GitHub/Next-Gen-App/build/ios_results_1696933915157.xcresult
        : Waiting for a connection from Flutter on iPhone 14 Plus...
        Command line invocation:
            /Users/lallu/Downloads/Xcode.app/Contents/Developer/usr/bin/xcodebuild test-without-building -xctestrun /Users/lallu/Documents/GitHub/Next-Gen-App/build/ios_integ/Build/Products/jk_iphonesimulator16.2-arm64-x86_64.xctestrun -only-testing RunnerUITests -destination "platform=iOS Simulator,name=iPhone 14 Plus" -resultBundlePath /Users/lallu/Documents/GitHub/Next-Gen-App/build/ios_results_1696933915157.xcresult
        
        User defaults from command line:
            IDEBuildOperationResultBundlePath = /Users/lallu/Documents/GitHub/Next-Gen-App/build/ios_results_1696933915157.xcresult
            IDEPackageSupportUseBuiltinSCM = YES
        
        Writing result bundle at path:
                /Users/lallu/Documents/GitHub/Next-Gen-App/build/ios_results_1696933915157.xcresult
        
        2023-10-10 16:01:57.989840+0530 RunnerUITests-Runner[5948:612402] [SceneConfiguration] Info.plist contained no UIScene configuration dictionary (looking for configuration named "(no name)")
        2023-10-10 16:01:58.000052+0530 RunnerUITests-Runner[5948:612402] Running tests...
        2023-10-10 16:01:59.107425+0530 RunnerUITests-Runner[5948:612402] PatrolServer: INFO: PATROL_PORT is null, falling back to default (8081)
        2023-10-10 16:01:59.107537+0530 RunnerUITests-Runner[5948:612402] PatrolServer: INFO: PatrolServer constructor called
        2023-10-10 16:01:59.107615+0530 RunnerUITests-Runner[5948:612402] PatrolServer: INFO: PATROL_ENABLED flag is defined
        2023-10-10 16:01:59.108806+0530 RunnerUITests-Runner[5948:612402] PatrolAppServiceClient: created, port: 8082
        2023-10-10 16:01:59.108836+0530 RunnerUITests-Runner[5948:612578] PatrolServer: INFO: Starting server...
            t =      nans Checking existence of `"Allow" Button`
        2023-10-10 16:01:59.111304+0530 RunnerUITests-Runner[5948:612578] PatrolServer: INFO: Server started on http://0.0.0.0:8081
        2023-10-10 16:01:59.111354+0530 RunnerUITests-Runner[5948:612578] Server loop done, error: (null)
            t =      nans Ignoring failure to get hierarchy for remote element in process 3856 (Error getting main window kAXErrorServerNotFound)
            t =      nans Open com.paytronix.nexgenfl
            t =      nans     Launch com.paytronix.nexgenfl
            t =      nans         Setting up automation session
            t =      nans         Wait for com.paytronix.nexgenfl to idle
        : Syncing files to device iPhone 14 Plus...                           7.7s
        : 
Hot Restart: attached to the app (press "r" to restart)
Hot Restart: logs are not connected yet
        : Flutter run key commands.
        : r Hot reload. 🔥🔥🔥
        : R Hot restart.
        : h List all available interactive commands.
        : d Detach (terminate "flutter run" but leave application running).
        : c Clear the screen
        : q Quit (terminate the application on the device).
        : 
        : A Dart VM Service on iPhone 14 Plus is available at: http://127.0.0.1:61396/AqaF9uFAzBg=/
        : The Flutter DevTools debugger and profiler on iPhone 14 Plus is available at: http://127.0.0.1:9100?uri=http://127.0.0.1:61396/AqaF9uFAzBg=/
Hot Restart: logs connected
        Showing iPhone 14 Plus logs:
Hot Restart for entrypoint test_bundle.dart...
        : 
        : Performing hot restart...                                       
        : Restarted application in 1,031ms.
        Patrol (native): initialize() started
        Patrol (native): initialize() succeeded
        00:00 +0: example_test counter state is the same after going to home and switching apps
        Patrol (native): configure() started
        Patrol (native): configure() succeeded
        Patrol (native): pressHome() started
        2023-10-10 16:02:19.572585+0530 RunnerUITests-Runner[5948:612402] PatrolServer: INFO: pressing home button...
            t =      nans Pressing Home button
        [en]
        en
        null
        2023-10-10 16:02:20.044754+0530 RunnerUITests-Runner[5948:612402] PatrolServer: INFO: done pressing home button
        2023-10-10 16:02:20.047033+0530 RunnerUITests-Runner[5948:612402] PatrolServer: INFO: result: ()
        Patrol (native): pressHome() succeeded
        ══╡ EXCEPTION CAUGHT BY FLUTTER FRAMEWORK ╞═════════════════════════════════════════════════════════
        The following assertion was thrown while executing callbacks for FrameTiming:
        'package:flutter/src/widgets/binding.dart': Failed assertion: line 881 pos 16: 'sendFramesToEngine':
        is not true.
        flutter:
        Either the assertion indicates an error in the framework itself, or we should provide substantially
        more information in this error message to help you determine and fix the underlying cause.
        In either case, please report this assertion by filing a bug on GitHub:
          https://github.com/flutter/flutter/issues/new?template=2_bug.md
        flutter:
        When the exception was thrown, this was the stack:
        #2      WidgetsBinding.drawFrame.<anonymous closure> (package:flutter/src/widgets/binding.dart:881:16)
        #3      SchedulerBinding._executeTimingsCallbacks (package:flutter/src/scheduler/binding.dart:329:19)
        #4      _invoke1 (dart:ui/hooks.dart:158:13)
        #5      PlatformDispatcher._reportTimings (dart:ui/platform_dispatcher.dart:555:5)
        #6      _reportTimings (dart:ui/hooks.dart:107:31)
        (elided 2 frames from class _AssertionError)
        flutter:
        The TimingsCallback that gets executed was:
          Closure: (List<FrameTiming>) => void
        ════════════════════════════════════════════════════════════════════════════════════════════════════
        00:00 +0: example_test counter state is the same after going to home and switching apps [E]
          Test failed. See exception logs above.
          The test description was: counter state is the same after going to home and switching apps
        flutter:
        00:00 +0 -1: (tearDownAll)
        00:00 +1 -1: Some tests failed.
Hot Restart for entrypoint test_bundle.dart...
        : 
        : Performing hot restart...                                       
        : Restarted application in 892ms.
        Patrol (native): initialize() started
        Patrol (native): initialize() succeeded
        00:00 +0: example_test counter state is the same after going to home and switching apps
        Patrol (native): configure() started
        Patrol (native): configure() succeeded
        Patrol (native): pressHome() started
        2023-10-10 16:02:51.741735+0530 RunnerUITests-Runner[5948:612402] PatrolServer: INFO: pressing home button...
            t =      nans Pressing Home button
        [en]
        en
        null
        2023-10-10 16:02:52.204464+0530 RunnerUITests-Runner[5948:612402] PatrolServer: INFO: done pressing home button
        2023-10-10 16:02:52.204596+0530 RunnerUITests-Runner[5948:612402] PatrolServer: INFO: result: ()

Patrol version

patrol_cli v2.2.0

Patrol Doctor output

Program adb found in /Users/lallu/Documents/platform-tools/adb
Env var $ANDROID_HOME set to /Users/lallu/Library/Android/sdk
Program xcodebuild found in /usr/bin/xcodebuild
Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller

Flutter Doctor output

[✓] Flutter (Channel stable, 3.10.6, on macOS 13.3 22E252 darwin-arm64, locale en-IN)
[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    ! Some Android licenses not accepted. To resolve this, run: flutter doctor --android-licenses
[✓] Xcode - develop for iOS and macOS (Xcode 14.2)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.3)
[✓] VS Code (version 1.83.0)
[✓] Connected device (3 available)
[✓] Network resources

[prasannavenkateshpalanisamy]

Kindly check the xcode configuration for my flavor - jk. Please ping in case of any configuration issues in the xcode

[balajiks-dev]

Same issue for me. Any solution for this.

[lravihs]

same issue for me @bartekpacia . Any solutions pls.?

[bartekpacia]

Is the problem that, during Hot Restart, if you tap on the app with a finger/mouse, it turns black with a violet text "Test starting/finished"?

[prasannavenkateshpalanisamy]

No need to tap anything. Once pressing 'r' for Hot restart the test starting screen will appear and the tests will execute.

[prasannavenkateshpalanisamy]

we have flavors for our application. It will be better if you tell us the configuration for the flavors in xcode

[jacjac11]

same here, works for android but not ios. In gitlab we can't wait and execute restart manually, is there a workaround?
patrol: ^2.4.0-dev.3
flutter version 3.13.8

Hot Restart: logs connected
Hot Restart: not attached to the app yet
Showing iPhone 14 logs:
: Syncing files to device iPhone 14... 23.6s
:
Hot Restart: attached to the app (press "r" to restart)
: Flutter run key commands.
: r Hot reload. 🔥🔥🔥
: R Hot restart.
: h List all available interactive commands.
: d Detach (terminate "flutter run" but leave application running).
: c Clear the screen
: q Quit (terminate the application on the device).
:
: A Dart VM Service on iPhone 14 is available at: http://127.0.0.1:54802/0MTMq53C8qo=/
: The Flutter DevTools debugger and profiler on iPhone 14 is available at: xxx
Hot Restart for entrypoint test_bundle.dart...
:
: Performing hot restart...
: Restarted application in 2,280ms.

After test passed, it shows test passed but on terminal it didn't reach a complete stage ( probably due to patrol test command)?

@bartekpacia

[bartekpacia]

I reproduced this problem with the application that @lravihs shared with LeanCode over private channels. Thank you all.

[jacjac11]

We got it to work by

1. Create a new flavor for eg: integrationtest
2. At Targets > Runner > Build Settings, find FLUTTER_TARGET, set the new flavor entry point to integration_test/test_bundle.dart

[lravihs]

@jacjac11 @bartekpacia thanks a lot. It worked finally for us. It's the solution here

[prasannavenkateshpalanisamy]

The solution got worked and hence closing this ticket.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar problem, please file a new issue. Make sure to follow the template and provide all the information necessary to reproduce the issue.