Skip to content

Commit

Permalink
Sync integration tests (#3690)
Browse files Browse the repository at this point in the history
* Upgraded RealmAppImport to use the realm-cli

* Changed to symlinked consumation for the electron app

* Upgraded mocha-remote and fixed integration tests

* Migrated to importing from "realm"

* Major refactoring to the integration test harnesses

* Simplified the react-native integration test app

* Made the import app utility return an app instance

* Adding a failing anonymous credentials test

* Attempt at running integration tests on CI

* Switched to lerna when installing environments and tests

* Updating some NPM scripts

* Fixing root package-lock.json

* Removed unneeded complexity in RN test app.

* Ignoring scripts when installing locally

* Upgraded typescript and package-locks

* Removed the regular dependency on realm and tests again

* Revert "Removed unneeded complexity in RN test app."

This reverts commit 16e04eb.

* Fixed running ios tests via a --prefix

* Updated the README

* Run linting on sub-packages explicitly

* Scoping bootstrapping in GH workflow

* Propagating context as environments

* Adding skipIf to suites

* Running local only integration tests on CI

* Using lerna bootstrap instead of npm install

* Renamed localOnly to missingServer

* Moved integration tests to GHA

* Running iOS integration tests

* Caching xcframework + installing tests package

* Inhibiting warnings

* Using the mocha-github-actions-reporter

* Run in chrome-debugging mode

* Upgrading mocha remote

* Switch mode from context

* Updated script seperator to :

* No need to disconnect on end with autoDisconnect

* Caching XCode derived data

* Caching CocoaPods

* Include the CONTEXT environment variable in NPM script

* Try running just the Chrome Debugging

* Adding cache on Podfile.lcok

* Installing watchman

* Updated .lock pattern

* Adding start.js to pass arguments to mocha

* Debugging metro cache invalidation

* Adding an option to run chrome debugger headless

* Printing module verboseNames

* Logging stack on error

* Try / catching more.

* Cache the ios build folder

* Iterating on printing error stacks

* Removed caching ios build artifacts again

* Logging a stack when a message is logged

* Caching node_modules

* Try messing less with the RN cache

* Retrying cache of the ios build folder

* Debugging the tests

* Adding a tmate session for debugging

* Trying to catch error loading "realm"

* Debugging requiring "realm"

* Reworded the install script override

* Removed peer dependencies on "realm" and the tests

* Bumping the ios-build cache version

* No need for a mocha peer dependency

* Re-introducing the tests

* Propagate the error through

* Adding build:watch script to tests

* Try running tests on Android emulator

* Building and running tests for android

* Run on Electron as well

* Reexporting ANDROID_NDK_HOME

* Using xvfb-run

* Simplified electron harness

* Trying Android test again

* Trying a node matrix

* Update pr-integration-tests.yml

* Adding windows to the Node.js builds

* Adding steps to setup windows build env

* Use prebuilds for node and run in release and debug mode

* Adding runtime and target to package.json's config

* Upgraded mocha-remote-cli

* Retry opening the Metro Chrome debugging UI

* Run React Native in both release & debug

* Fixed RN Android job name

* Renamed flow file and name

* Upgrading mocha-remote

* Upgrading CocoaPods and Flipper-Folly

* Calling logcat before exiting on Android

* Updated README.md files

* Adding a Realm.Set when chrome debugging (#3704)

* Enable streaming logcat while testing

* Clarified some documentation

* Minor clean ups

* Xcode updated some project files

* Attempting to apply a fix to breaking ios builds

* Simplified skipIf

* Moved the local_prebuilds env

* Reusing a variable

* Disabled chrome debugging tests for now

* Update CHANGELOG.md

* Apply suggestions from code review

Co-authored-by: Andrew Meyer <[email protected]>

Co-authored-by: Andrew Meyer <[email protected]>
  • Loading branch information
kraenhansen and takameyer committed Jun 10, 2021
1 parent 5b0eabf commit c31ea11
Show file tree
Hide file tree
Showing 117 changed files with 27,428 additions and 19,121 deletions.
6 changes: 5 additions & 1 deletion .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,8 @@ node_modules/
vendor/

/tests/test-runners/
/object-server-for-testing/
/object-server-for-testing/

# The sub-packages have their own linting
/integration-tests/
/packages/
240 changes: 240 additions & 0 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,240 @@
name: Integration Tests

on:
pull_request:
paths:
# Source code
- '*'
- 'lib/**'
- 'src/**'
- 'types/**'
- 'react-native/**'
- 'vendor/**'
# Integration tests
- 'integration-tests/**'
# No need to run when updating documentation
- '!**.md'

jobs:
node:
name: Node.js v${{ matrix.node }} on ${{ matrix.os }} (${{ matrix.type }})
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
node: [12, 14]
type: [release, debug]
steps:
- uses: actions/checkout@v2
with:
submodules: 'recursive'
- uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node }}
registry-url: https://registry.npmjs.org/
# The following is a noop on non-windows runners
- name: Install Microsoft Visual C++ cmd tools
uses: ilammy/msvc-dev-cmd@v1
- name: Windows Environment setup
if: ${{ matrix.os == 'windows-latest' }}
shell: powershell
run: |
Invoke-WebRequest -Uri https://static.realm.io/downloads/vcpkg.zip -OutFile C:\vcpkg.zip
Expand-Archive -Path C:\vcpkg.zip -DestinationPath C:\
# Install the root package (--ignore-scripts to avoid downloading or building the native module)
- run: npm ci --ignore-scripts
# Build the native module
- name: Prebuild release binary
run: npx prebuild
if: ${{ matrix.type == 'release' }}
- name: Prebuild debug binary
run: npx prebuild --debug
if: ${{ matrix.type == 'debug' }}
# Bootstrap lerna sub-packages (builds the packages and Realm JS native module)
- run: npx lerna bootstrap --scope realm-node-tests --include-dependencies
env:
npm_config_realm_local_prebuilds: ${{ github.workspace }}/prebuilds
# Run tests
- run: npm test --prefix integration-tests/environments/node
env:
MOCHA_REMOTE_CONTEXT: missingServer
MOCHA_REMOTE_REPORTER: mocha-github-actions-reporter
electron-linux:
name: Electron on Linux
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
submodules: 'recursive'
- uses: actions/setup-node@v1
with:
node-version: 12
registry-url: https://registry.npmjs.org/
# Install the root package (--ignore-scripts to avoid downloading or building the native module)
- run: npm ci --ignore-scripts
# Bootstrap lerna sub-packages (builds the packages and Realm JS native module)
- run: npx lerna bootstrap --scope realm-electron-tests --include-dependencies
# Run tests (using xvfb-run to enable opening windows in a virtual framebuffer)
- name: Run tests (Electron / Main)
run: xvfb-run npm run test:main --prefix integration-tests/environments/electron
env:
MOCHA_REMOTE_CONTEXT: missingServer
MOCHA_REMOTE_REPORTER: mocha-github-actions-reporter
- name: Run tests (Electron / Renderer)
run: xvfb-run npm run test:renderer --prefix integration-tests/environments/electron
env:
MOCHA_REMOTE_CONTEXT: missingServer
MOCHA_REMOTE_REPORTER: mocha-github-actions-reporter
react-native-ios:
name: React Native on iOS (${{ matrix.type }})
runs-on: macos-latest
strategy:
fail-fast: false
matrix:
type: [Release, Debug]
steps:
- uses: actions/checkout@v2
with:
submodules: 'recursive'
- uses: actions/setup-node@v1
with:
node-version: 12
registry-url: https://registry.npmjs.org/
# Setup tools
- name: Install tools
run: brew install watchman
# TODO: Expose a way to skip building the Node.js native module
- name: Patch package.json to avoid building Node.js native module
run: cat <<< $(jq '.scripts.install = "echo skipping building Node.js binary"' < package.json) > package.json

# Caches
- uses: actions/cache@v2
with:
path: ~/.npm
key: npm-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
npm-
- uses: actions/cache@v2
name: Restore iOS build folder from cache
with:
path: react-native/ios/build
key: ios-build-dir-${{ hashFiles('src/**', 'react-native/ios/**', 'vendor/**') }}
restore-keys: |
ios-build-dir-
- uses: actions/cache@v2
name: Restore xcframework from cache
id: cache-xcframework
with:
path: react-native/ios/realm-js-ios.xcframework
key: xcframework-${{ hashFiles('src/**', 'react-native/ios/**', 'vendor/**') }}
- uses: actions/cache@v2
name: Restore Xcode "derived data" from cache
with:
path: ~/Library/Developer/Xcode/DerivedData
key: xcode-derived-data
- uses: actions/cache@v2
name: Restore CocoaPods from cache
with:
path: integration-tests/environments/react-native/ios/Pods
key: cocoapods-${{ hashFiles('integration-tests/environments/react-native/ios/Podfile.lock') }}
restore-keys: | # Enables running from an old build
cocoapods-
- run: npm ci
# Build the iOS xcframework (if cache missed)
- run: ./scripts/build-ios.sh -c ${{ matrix.type }} -s
if: ${{ steps.cache-xcframework.outputs.cache-hit != 'true' }}
# Bootstrap lerna sub-packages (builds the packages, the Realm JS native module and pod install)
- run: npx lerna bootstrap --scope '{realm-integration-tests,realm-react-native-tests}' --include-dependencies
# Run the tests
- name: Run tests (iOS / Native)
run: npm run test:ios --prefix integration-tests/environments/react-native
env:
MOCHA_REMOTE_CONTEXT: missingServer
MOCHA_REMOTE_REPORTER: mocha-github-actions-reporter
MOCHA_REMOTE_EXIT_ON_ERROR: true
- name: Run tests (iOS / Chrome Debugging)
if: ${{ false }}
run: npm run test:ios:chrome --prefix integration-tests/environments/react-native
env:
MOCHA_REMOTE_CONTEXT: missingServer
MOCHA_REMOTE_REPORTER: mocha-github-actions-reporter
MOCHA_REMOTE_EXIT_ON_ERROR: true
HEADLESS_DEBUGGER: true
react-native-android:
name: React Native on Android (${{ matrix.type }})
runs-on: macos-latest
strategy:
fail-fast: false
matrix:
type: [Release, Debug]
steps:
- uses: actions/checkout@v2
with:
submodules: 'recursive'
- uses: actions/setup-node@v1
with:
node-version: 12
registry-url: https://registry.npmjs.org/
# Setup tools
- name: Install tools
run: brew install watchman
# TODO: Expose a way to skip building the Node.js native module
- name: Patch package.json to avoid building Node.js native module
run: cat <<< $(jq '.scripts.install = "echo skipping building Node.js binary"' < package.json) > package.json
# Install the root package
- uses: actions/cache@v2
name: Restore node_modules
id: cache-node-modules
with:
path: node_modules
key: node-modules-${{ hashFiles('package-lock.json') }}

# Caches
- uses: actions/cache@v2
name: Restore Android build folders from cache
with:
path: |
build-realm-android
react-native/android/build
key: android-build-dir-${{ hashFiles('src/**', 'react-native/android/**', 'vendor/**') }}
restore-keys: |
android-build-dir-
- uses: actions/cache@v2
name: Restore Android librealm.so from cache
id: cache-android-so
with:
path: react-native/android/**/librealm.so
key: android-so-${{ hashFiles('src/**', 'react-native/android/**', 'vendor/**') }}

- run: npm ci
if: ${{ steps.cache-node-modules.outputs.cache-hit != 'true' }}
# Build the Android shared object file (if cache missed)
- run: ANDROID_NDK=$ANDROID_NDK_HOME node scripts/build-android.js --arch=x86 --build-type=${{ matrix.type }}
if: ${{ steps.cache-android-so.outputs.cache-hit != 'true' }}
# Bootstrap lerna sub-packages (builds the packages, the Realm JS native module and pod install)
- run: npx lerna bootstrap --scope '{realm-integration-tests,realm-react-native-tests}' --include-dependencies
# Run the tests
- name: Run tests (Android / Native)
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: 29
script: npm run test:android --prefix integration-tests/environments/react-native
env:
MOCHA_REMOTE_CONTEXT: missingServer
MOCHA_REMOTE_REPORTER: mocha-github-actions-reporter
MOCHA_REMOTE_EXIT_ON_ERROR: true
SPAWN_LOGCAT: true
- name: Run tests (Android / Chrome Debugging)
if: ${{ false }}
uses: reactivecircus/android-emulator-runner@v2
with:
api-level: 29
script: npm run test:android:chrome --prefix integration-tests/environments/react-native
env:
MOCHA_REMOTE_CONTEXT: missingServer
MOCHA_REMOTE_REPORTER: mocha-github-actions-reporter
MOCHA_REMOTE_EXIT_ON_ERROR: true
HEADLESS_DEBUGGER: true
SPAWN_LOGCAT: true
2 changes: 1 addition & 1 deletion .github/workflows/pr-realm-web.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
# Install the root package (--ignore-scripts to avoid downloading or building the native module)
- run: npm ci --ignore-scripts
# Bootstrap lerna sub-packages
- run: npx lerna bootstrap
- run: npx lerna bootstrap --scope realm-web-integration-tests --include-dependencies
# Lint, build and test the package
- run: npm run lint
working-directory: packages/realm-web
Expand Down
9 changes: 7 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ node_modules/
npm-debug.log
build/
/prebuilds/
*.lock
yarn.lock

# Emacs
*~
Expand Down Expand Up @@ -66,10 +66,15 @@ examples/ReactExample/ios/ReactExample.xcworkspace/
tests/mongodb-realm

# Integration tests
integration-tests/**/*.tgz
integration-tests/environments/electron/dist/
integration-tests/environments/react-native/ios/Pods/
integration-tests/tests/dist/
# TODO: Solve .lock files are produced in "throws when called with an empty string"
integration-tests/**/.lock

# Realm App Importer artifacts
**/imported-apps/
**/realm-config

#Ignore symlinked directories (otherwise on Windows they are shown as untracked files)
/react-native/android/src/main/jni/vendor/
Expand Down
1 change: 0 additions & 1 deletion .watchmanconfig
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
"node_modules",
"react-native/node_modules",
"packages",
"integration-tests",
"tests"
]
}
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,8 @@ NOTE: This release has a number of fixes compared to v10.3.0-rc.1. For a complet
* Upgraded Realm Core from v10.6.0 to 10.7.1
* Switch to unified releases of Realm Core, Realm Sync and Realm Object Store.
* Upgraded to Realm Core v10.5.4.
* Improved the integration test harness to increase developer experience, enable tests needing a server and importing Realm apps on demand. ([#3690](https://github.com/realm/realm-js/pull/3690))
* Migrated integration tests to GitHub Actions. ([#3690](https://github.com/realm/realm-js/pull/3690))

10.2.0 Release notes (2021-2-5)
=============================================================
Expand Down
Loading

0 comments on commit c31ea11

Please sign in to comment.