[IOS] Failing to execute tests in Bitrise for debug variant #819

GuiCavi · 2018-07-10T21:21:30Z

Description

I'm using Bitrise to run Detox tests. The build command runs ok and is a success. But the test command always fails.

Running locally:

Command detox test -c ios.sim.debug without starting the packager: 'No bundle URL present' error
Command detox test -c ios.sim.debug with the packager: Opens the packager and everything runs OK.
Command detox test -c ios.sim.release without the packager: works fine.
Command detox test -c ios.sim.release with the packager: works fine.

Running in Bitrise:

Command detox test -c ios.sim.debug --cleanup without starting the packager: 'No bundle URL present' error
Command detox test -c ios.sim.debug --cleanup with the packager:

Verify tests are up
    "before all" hook:
    Error: Timeout of 120000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/Users/vagrant/git/e2e/tests.spec.js)

Command detox test -c ios.sim.release --cleanup without the packager: works fine.
Command detox test -c ios.sim.release --cleanup with the packager: works fine.

Disclaimer
With/Without the packager means the variable RCT_NO_LAUNCH_PACKAGER equals false/true, respectively.

The package.json file

"ios.sim.debug": {
    "binaryPath": "ios/build/Build/Products/Debug-iphonesimulator/PROJECT_NAME.app",
    "build": "export RCT_NO_LAUNCH_PACKAGER=true && xcodebuild -workspace ios/PROJECT_NAME.xcworkspace -scheme INTEGRATION_SCHEME -configuration Debug -sdk iphonesimulator -derivedDataPath ios/build",
    "type": "ios.simulator",
    "name": "iPhone 7"
},
"ios.sim.release": {
    "binaryPath": "ios/build/Build/Products/Release-iphonesimulator/PROJECT_NAME.app",
    "build": "export RCT_NO_LAUNCH_PACKAGER=true && xcodebuild -workspace ios/PROJECT_NAME.xcworkspace -scheme PRODUCTION_SCHEME -configuration Release -sdk iphonesimulator -derivedDataPath ios/build",
    "type": "ios.simulator",
    "name": "iPhone 7 Plus"
},

The bitrise.yml file

_init_install:
    steps:
    - [email protected]:
        run_if: '{{getenv "SSH_RSA_PRIVATE_KEY" | ne ""}}'
    - [email protected]: {}
    - [email protected]:
        inputs:
        - content: |-
            #!/usr/bin/env bash
            # fail if any commands fails
            set -e
            # debug log
            set -x

            # write your script here
            brew uninstall --ignore-dependencies node
            brew install node@8
            brew link --overwrite --force node@8
    - [email protected]: {}

_detox_tests:
    steps:
    - [email protected]:
        title: Install Detox CLI
        inputs:
        - command: install -g detox-cli
    - [email protected]:
        title: Install React-Native CLI
        inputs:
        - command: install -g react-native-cli
    - [email protected]:
        title: Install Detox Utils
        inputs:
        - content: |-
            #!/bin/bash
            brew tap facebook/fb
            export CODE_SIGNING_REQUIRED=NO
            brew install fbsimctl
            brew tap wix/brew
            brew install applesimutils --HEAD
    - [email protected]:
        title: Detox - Build Debug App
        inputs:
        - content: |-
            #!/usr/bin/env bash
            detox build -c ios.sim.debug
    - [email protected]:
        title: Detox - Run E2E Tests
        inputs:
        - content: |-
            #!/usr/bin/env bash
            detox test -c ios.sim.debug --cleanup
    - [email protected]: {}

test_ios:
    before_run:
    - _init_install
    after_run:
    - _detox_tests
    steps:
    - [email protected]:
        inputs:
        - podfile_path: ios/Podfile
    - [email protected]:
        inputs:
        - extract_to_path: "$HOME/Documents/FacebookSDK/"
        - archive_url: https://origincache.facebook.com/developers/resources/?id=facebook-ios-sdk-current.zip
    - [email protected]: {}

Detox, Node, Device, Xcode and macOS Versions

Detox: v7.4.3
React Native: v0.51.0
Node: v9.4.0
Device: iPhone 7 (simulator)
Xcode: Version 9.3 (9E145)
macOS: 10.13.3 (17D102)

Device and verbose Detox logs

Error detox test -c ios.sim.debug --cleanup with the packager

Verify tests are up
detox info 12:14:57: server listening on localhost:49300...
detox info 2: Searching for device matching iPhone 7... 
detox info 5: Booting device 923035AA-8378-47FE-A4FE-CE5205AF323C 
detox info 7: Uninstalling com.PROJECT_NAME.PROJECT_NAME... 
detox info 7: com.PROJECT_NAME.PROJECT_NAME uninstalled 
detox info 8: Installing /Users/vagrant/git/ios/build/Build/Products/Debug-iphonesimulator/PROJECT_NAME.app... 
detox info 8: /Users/vagrant/git/ios/build/Build/Products/Debug-iphonesimulator/PROJECT_NAME.app installed 
detox info 9: Terminating com.PROJECT_NAME.PROJECT_NAME... 
detox info 9: com.PROJECT_NAME.PROJECT_NAME terminated 
detox info 10: Trying to set permissions... 
detox info 10: Permissions are set 
detox info 11: Launching com.PROJECT_NAME.PROJECT_NAME... 
detox info 11: com.PROJECT_NAME.PROJECT_NAME launched. The stdout and stderr logs were recreated, you can watch them with:
        tail -F $HOME/Library/Developer/CoreSimulator/Devices/923035AA-8378-47FE-A4FE-CE5205AF323C/data/tmp/detox.last_launch_app_log.{out,err} 
    1) "before all" hook
  0 passing (2m)
  1 failing
  1) Verify tests are up
       "before all" hook:
     Error: Timeout of 120000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. (/Users/vagrant/git/e2e/tests.spec.js)
  
detox info 12: Shutting down 923035AA-8378-47FE-A4FE-CE5205AF323C... 
detox info 12: 923035AA-8378-47FE-A4FE-CE5205AF323C shut down 
child_process.js:644
    throw err;
    ^
Error: Command failed: node_modules/.bin/mocha e2e --opts e2e/mocha.opts --configuration ios.sim.debug  --cleanup   --grep :android: --invert  
    at checkExecSyncError (child_process.js:601:13)
    at Object.execSync (child_process.js:641:13)
    at runMocha (/Users/vagrant/git/node_modules/detox/local-cli/detox-test.js:114:6)
    at run (/Users/vagrant/git/node_modules/detox/local-cli/detox-test.js:75:7)
    at Object.<anonymous> (/Users/vagrant/git/node_modules/detox/local-cli/detox-test.js:181:1)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)

File detox.last_launch_app_log.err prints:

Failed to load bundle(http://localhost:8081/index.bundle?platform=ios&dev=true&minify=false) with error:(Could not connect to development server.
Ensure the following:
- Node server is running and available on the same network - run 'npm start' from react-native root
- Node server URL is correctly set in AppDelegate
- WiFi is enabled and connected to the same network as the Node Server
URL: http://localhost:8081/index.bundle?platform=ios&dev=true&minify=false The operation couldn’t be completed. (NSURLErrorDomain error -999.))

TL;DR

I get an error when running Detox with Bitrise. The command detox test -c ios.sim.debug --cleanup should open a packager but looks like it's not doing it.

The version that is needed to test is the Debug one, no other options. Maybe I missed something about the installation, but all the files were checked with official docs and other examples and are the identical.

Related issues: #742, #749, #740

The text was updated successfully, but these errors were encountered:

pkantsedalov · 2018-07-24T10:31:31Z

@GuiCavi Hi. I have the similar issue.
How can it possible at all to run detox test without a packager? I mean is it possible to at first run detox build and then immediately run detox test?

GuiCavi · 2018-07-24T11:58:32Z

@pkantsedalov Hi, the only possible way to do that is by running a release version. This version doesn't need the packager. I'm not sure how Bitrise works internally, but I believe it:

Runs the build command
The build triggers the packager to open
It closes by the end of the command
Then it runs the test

The point is that the packager closes right before the tests starts, while running on a local machine it stays opened in another terminal window. Anyway, a workaround is to run the tests in Release version, pointing to a Test database (if needed).

What do you think of that?

pkantsedalov · 2018-07-25T07:53:55Z

@GuiCavi I arranged the next flow:

detox build -c {configurationFromPackageJson}
node_modules/react-native/local-cli/cli.js bundle --dev false --entry-file index.js --platform ios --reset-cache --bundle-output Build/Products/Debug-iphonesimulator/{myApp}.app/main.jsbundle --assets-dest Build/Products/Debug-iphonesimulator/{appName}.app
detox test -c {configurationFromPackageJson}

Works fine.

GuiCavi · 2018-07-25T13:29:40Z

That makes sense, I'll give it a try. All in the same step? Like:

- [email protected]:
        inputs:
        - content: |-
            #!/usr/bin/env bash
            # fail if any commands fails
            set -e
            # debug log
            set -x

            detox build -c {configurationFromPackageJson}
            node_modules/react-native/local-cli/cli.js bundle --dev false --entry-file index.js --platform ios --reset-cache --bundle-output Build/Products/Debug-iphonesimulator/{myApp}.app/main.jsbundle --assets-dest Build/Products/Debug-iphonesimulator/{appName}.app
            detox test -c {configurationFromPackageJson}

pkantsedalov · 2018-07-25T13:52:10Z

@GuiCavi sorry we don't have any CI solution for that yet. For now we have such script for local tests run.

stale · 2018-09-08T14:11:55Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.
Thank you for your contributions.

stale · 2018-09-15T14:21:53Z

The issue has been closed for inactivity.

noomorph · 2018-09-18T11:34:05Z

From what I see, it could be good if we were running Detox e2e test builds in debug mode as well, preferably including Bitrise. I think it needs to be filled as a separate issue though.

stale bot added the 🏚 stale label Sep 8, 2018

stale bot closed this as completed Sep 15, 2018

noomorph removed the 🏚 stale label Sep 18, 2018

lock bot locked as resolved and limited conversation to collaborators Sep 21, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[IOS] Failing to execute tests in Bitrise for debug variant #819

[IOS] Failing to execute tests in Bitrise for debug variant #819

GuiCavi commented Jul 10, 2018 •

edited

Loading

pkantsedalov commented Jul 24, 2018

GuiCavi commented Jul 24, 2018

pkantsedalov commented Jul 25, 2018 •

edited

Loading

GuiCavi commented Jul 25, 2018

pkantsedalov commented Jul 25, 2018

stale bot commented Sep 8, 2018

stale bot commented Sep 15, 2018

noomorph commented Sep 18, 2018

[IOS] Failing to execute tests in Bitrise for debug variant #819

[IOS] Failing to execute tests in Bitrise for debug variant #819

Comments

GuiCavi commented Jul 10, 2018 • edited Loading

Description

The package.json file

The bitrise.yml file

Detox, Node, Device, Xcode and macOS Versions

Device and verbose Detox logs

TL;DR

pkantsedalov commented Jul 24, 2018

GuiCavi commented Jul 24, 2018

pkantsedalov commented Jul 25, 2018 • edited Loading

GuiCavi commented Jul 25, 2018

pkantsedalov commented Jul 25, 2018

stale bot commented Sep 8, 2018

stale bot commented Sep 15, 2018

noomorph commented Sep 18, 2018

GuiCavi commented Jul 10, 2018 •

edited

Loading

pkantsedalov commented Jul 25, 2018 •

edited

Loading