Merge branch 'master' into kh/realm-core-after-11.4.1

.github/pull_request_template.md

@@ -9,6 +9,7 @@ This closes # ???
 * [ ] 📝 Changelog entry
 * [ ] 📝 `Compatibility` label is updated or copied from previous entry
 * [ ] 🚦 Tests
+* [ ] 📱 Check the React Native/other sample apps work if necessary
 * [ ] 📝 Public documentation PR created or is not necessary
 * [ ] 💥 `Breaking` label has been applied or is not necessary
 

.github/workflows/integration-tests.yml

@@ -33,11 +33,12 @@ jobs:
       fail-fast: false
       matrix:
         os:
-          [
-            { name: Linux, runner: ubuntu-latest },
-            { name: Window, runner: windows-latest },
-            { name: macOS, runner: macos-latest },
-          ]
+          - name: Linux
+            runner: ubuntu-latest
+          - name: Window
+            runner: windows-latest
+          - name: macOS
+            runner: macos-latest
         node: [14, 16]
         type: [release, debug]
     steps:
@@ -155,10 +156,13 @@ jobs:
         #type: [Release, Debug]
         type: [Release]
         platform:
-          [
-            { name: ios, build-configuration: simulator },
-            { name: catalyst, build-configuration: catalyst },
-          ]
+          - name: ios
+            build-configuration: simulator
+          - name: catalyst
+            build-configuration: catalyst
+    env:
+      # Pin the Xcode version
+      DEVELOPER_DIR: /Applications/Xcode_12.5.1.app
     steps:
       - uses: actions/checkout@v2
         with:

.github/workflows/package-unit-tests.yml

@@ -0,0 +1,34 @@
+name: Package Unit Tests
+
+env:
+  REALM_DISABLE_ANALYTICS: 1
+  MOCHA_REMOTE_TIMEOUT: 10000
+
+on:
+  pull_request:
+    paths:
+      # Source code
+      - "packages/realm-network-transport/**"
+      - "packages/realm-common/**"
+      # No need to run when updating documentation
+      - "!**.md"
+      # Run this on updates to the workflow
+      - ".github/workflows/package-unit-tests.yml"
+
+jobs:
+  unit-tests-linux:
+    name: Unit tests on Linux
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          submodules: "recursive"
+      - uses: actions/setup-node@v1
+        with:
+          node-version: 14
+          registry-url: https://registry.npmjs.org/
+      - name: Install npm v7
+        run: npm install -g npm@7
+      - run: npx lerna bootstrap --scope '{realm-network-transport,@realm.io/common}' --include-dependencies
+      - run: npm test --prefix packages/realm-network-transport
+      - run: npm test --prefix packages/realm-common

.vscode/launch.json

@@ -85,7 +85,7 @@
                 "--expose_gc",
                 "${workspaceFolder}/tests/node_modules/jasmine/bin/jasmine.js",
                 "spec/unit_tests.js",
-                "--filter=."
+                "--filter=${input:testFilter}"
             ],
             "cwd": "${workspaceFolder}/tests"
         },
@@ -105,5 +105,13 @@
                 "Attach to Port"
             ]
         }
+    ],
+    "inputs": [
+        {
+            "id": "testFilter",
+            "type": "promptString",
+            "default": ".",
+            "description": "Filtering used to limit what tests are run"
+        }
     ]
 }

.watchmanconfig

@@ -1,8 +1,9 @@
 {
   "ignore_dirs": [
-    "node_modules",
     "react-native/node_modules",
-    "packages",
+    "packages/realm-app-importer",
+    "packages/realm-web-importer",
+    "packages/realm-web-integration-tests",
     "tests"
   ]
 }

CHANGELOG.md

@@ -1,7 +1,13 @@
 x.x.x Release notes (yyyy-MM-dd)
 =============================================================
+
+### Deprecations
+* `EmailPasswordAuth` method calls using positional arguments are being deprecated in favour of using a single object dictionary argument. See "Examples of how to update to the new API" in the description of https://github.com/realm/realm-js/pull/4041 for examples of how to update each method call.
+    * The existing methods will be removed in the next major version.
+    * The existing methods will continue to work but will output a deprecation warning to the console in development mode when used, of the form: `Deprecation warning from Realm: resetPassword(password, token, token_id) is deprecated and will be removed in a future major release. Consider switching to resetPassword({ password, token, token_id })`.
+
 ### Enhancements
-* None.
+* New consistent API for `EmailPasswordAuth` methods, using a single object dictionary rather than positional arguments, to fix inconsistencies and make usage clearer. The existing API is being deprecated (see above). ([#3943](https://github.com/realm/realm-js/issues/3943))
 
 ### Fixed
 * Using `sort`, `distinct`, or `limit` as property name in query expression would cause an `Invalid predicate` error. ([realm/realm-java#7545](https://github.com/realm/realm-java/issues/7545), since v10.0.2)
@@ -18,11 +24,14 @@ x.x.x Release notes (yyyy-MM-dd)
 * File format: generates Realms with format v22 (reads and upgrades file format v5 or later for non-synced Realm, upgrades file format v10 or later for synced Realms).
 
 ### Internal
-* Adding use of [ccache](https://ccache.dev/) in build scripts, XCode projects and the integration tests GHA workflow.
 * Upgraded Realm Core from v11.4.1 to v10.6.0.
+* Adding use of [ccache](https://ccache.dev/) in build scripts, XCode projects and the integration tests GHA workflow.
 * Change Apple/Linux temporary directory to default to the environment's `TMPDIR` if available. This is primarily used by tests. ([realm/realm-core#4921](https://github.com/realm/realm-core/issues/4921))
 * Minor speed improvement in property setters. ([#4058](https://github.com/realm/realm-js/pull/4058) and [realm/realm-core#5011](https://github.com/realm/realm-core/pull/5011))
 * Adding use of [ccache](https://ccache.dev/) in build scripts, XCode projects and the integration tests GHA workflow.
+* Dropped using `install-local` in the integration tests, in favour of a more involved Metro configuration.
+* Adding combined type definition for `Realm.objects()` and `Realm.objectForPrimaryKey()` so they can be cleanly wrapped.
+* Adding colorized compiler diagnostics if using Ninja.
 
 10.9.1 Release notes (2021-10-20)
 =============================================================

CMakeLists.txt

@@ -1,3 +1,5 @@
+include(CheckCXXCompilerFlag)
+
 # compiling for Node.js, need to set up toolchains before project() call
 if(DEFINED CMAKE_JS_VERSION)
     if(CMAKE_HOST_SYSTEM_NAME STREQUAL "Windows")
@@ -35,6 +37,21 @@ if(CCACHE_PROGRAM)
     set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE "${CCACHE_PROGRAM}")
 endif()
 
+# Copied from Realm Core's CMakeList.txt
+function(add_cxx_flag_if_supported flag)
+    if(flag MATCHES "^-Wno-")
+        # Compilers ignore unknown -Wno-foo flags, so look for -Wfoo instead.
+        string(REPLACE "-Wno-" "-W" check_flag ${flag})
+    else()
+        set(check_flag ${flag})
+    endif()
+
+    check_cxx_compiler_flag(${check_flag} HAVE${check_flag})
+    if(HAVE${check_flag})
+        add_compile_options($<$<COMPILE_LANGUAGE:CXX>:${flag}>)
+    endif()
+endfunction()
+
 set(CMAKE_CXX_STANDARD 17)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
 set(CMAKE_CXX_EXTENSIONS OFF)
@@ -49,6 +66,13 @@ if(DEFINED CMAKE_JS_VERSION)
     include(NodeJSTargets)
 endif()
 
+if (NOT MSVC)
+    # Ninja buffers output so we need to tell the compiler to use colors even though stdout isn't a tty.
+    if(CMAKE_GENERATOR STREQUAL "Ninja")
+        add_cxx_flag_if_supported(-fdiagnostics-color)
+    endif()
+endif()
+
 option(REALM_JS_BUILD_CORE_FROM_SOURCE "Build Realm Core from source, as opposed to downloading prebuilt binaries" ON)
 if(REALM_JS_BUILD_CORE_FROM_SOURCE)
     set(REALM_BUILD_LIB_ONLY ON)

contrib/building.md

@@ -29,26 +29,27 @@
    * [Debugging the tests](#debugging-the-tests)
       * [Debugging React Native tests](#debugging-react-native-tests)
       * [Debugging Node.js tests using Visual Studio Code](#debugging-nodejs-tests-using-visual-studio-code)
+      * [Debugging failing Github Actions CI tests](#debugging-failing-github-actions-ci-tests)
    * [Testing against real apps](#testing-against-real-apps)
 
-<!-- Added by: tom.duncalf, at: Thu  7 Oct 2021 17:24:59 BST -->
+<!-- Added by: tom.duncalf, at: Thu  4 Nov 2021 12:06:47 GMT -->
 
 <!--te-->
 
 ## Pre-Requisites
 
 The following dependencies are required. All except Xcode can be installed by following the [setup instructions for MacOS section](#setup-instructions-for-macos).
 
-* [Xcode](https://developer.apple.com/xcode/) 12+ with Xcode command line tools installed
+- [Xcode](https://developer.apple.com/xcode/) 12+ with Xcode command line tools installed
   - Newer versions may work but 12.2 is the current recommended version, which can be downloaded from [Apple](https://developer.apple.com/download/all/?q=xcode%2012.2)
-* [Node.js](https://nodejs.org/en/) version 10.19 or later
+- [Node.js](https://nodejs.org/en/) version 10.19 or later
   - Consider [using NVM](https://github.com/nvm-sh/nvm#installing-and-updating) to enable fast switching between Node.js & NPM versions
-* [CMake](https://cmake.org/)
-* [OpenJDK 8](https://openjdk.java.net/install/)
-* [Android SDK 23+](https://developer.android.com/studio/index.html#command-tools)
-  -  Optionally, you can install [Android Studio](https://developer.android.com/studio)
-* [Android NDK 21.0](https://developer.android.com/ndk/downloads/index.html)
-* [Android CMake](https://developer.android.com/ndk/guides/cmake)
+- [CMake](https://cmake.org/)
+- [OpenJDK 8](https://openjdk.java.net/install/)
+- [Android SDK 23+](https://developer.android.com/studio/index.html#command-tools)
+  - Optionally, you can install [Android Studio](https://developer.android.com/studio)
+- [Android NDK 21.0](https://developer.android.com/ndk/downloads/index.html)
+- [Android CMake](https://developer.android.com/ndk/guides/cmake)
 
 ### Setup instructions for MacOS
 
@@ -171,12 +172,12 @@ Note: If you have cloned the repo previously make sure you remove your `node_mod
 
 ### Building for iOS
 
-* Run `./scripts/build-ios.sh` from the `realm-js` root directory
+- Run `./scripts/build-ios.sh` from the `realm-js` root directory
 
 ### Building for Android
 
-* Run `node scripts/build-android.js` from the `realm-js` root directory
-  -  The compiled version of the Android module is output to `<project-root>/android`
+- Run `node scripts/build-android.js` from the `realm-js` root directory
+  - The compiled version of the Android module is output to `<project-root>/android`
 
 ### Building for Node.js
 
@@ -190,26 +191,26 @@ If you want to build for Apple Silicon on an Intel based Mac, you can use the fo
 
 ```sh
  npm run build-m1
- ```
+```
 
 #### Additional steps for Windows
 
 On Windows you will need to setup the environment for node-gyp:
 
-* Option 1: Install windows-build-tools Node.js package
+- Option 1: Install windows-build-tools Node.js package
 
-    ```cmd
-    # run in elevated command prompt (as Administrator)
-    npm install -g --production windows-build-tools
-    ```
+  ```cmd
+  # run in elevated command prompt (as Administrator)
+  npm install -g --production windows-build-tools
+  ```
 
-* Option 2: Manually install and configure as described in the [node-gyp](https://github.com/nodejs/node-gyp) manual.
+- Option 2: Manually install and configure as described in the [node-gyp](https://github.com/nodejs/node-gyp) manual.
 
-    Note you may need to configure the build tools path using npm
+  Note you may need to configure the build tools path using npm
 
-    ```cmd
-    npm config set msbuild_path "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\MSBuild.exe"
-    ```
+  ```cmd
+  npm config set msbuild_path "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\MSBuild.exe"
+  ```
 
 You also need to install openssl libraries with vcpkg:
 
@@ -227,7 +228,7 @@ copy .\packages\openssl-windows_x64-windows-static\lib\ssleay32.lib C:\src\vcpkg
 
 API documentation is written using [JSDoc](http://usejsdoc.org/). To generate the documentation, run:
 
-```npm run jsdoc```
+`npm run jsdoc`
 
 The generated docs can be found in `docs/output/realm/<version>/index.html`.
 
@@ -255,10 +256,10 @@ See [the instructions in the `integration-tests`](../integration-tests/README.md
 
 To run the the tests, run the `scripts/test.sh` script, passing an argument for which tests you would like to execute. The following options are available:
 
-* `react-tests` - runs all React Native tests on iOS Simulator
-* `react-tests-android` - runs all React Native Android tests on Android emulator
-* `node` - runs all tests for Node.js
-* `test-runners` - checks supported tests runners are working correctly
+- `react-tests` - runs all React Native tests on iOS Simulator
+- `react-tests-android` - runs all React Native Android tests on Android emulator
+- `node` - runs all tests for Node.js
+- `test-runners` - checks supported tests runners are working correctly
 
 For example:
 
@@ -273,6 +274,7 @@ Run `npm run lint` to lint the source code using `eslint`.
 #### Testing on Windows
 
 On Windows some of these targets are available as npm commands.
+
 ```
 npm run eslint
 npm run node-tests
@@ -304,22 +306,37 @@ You can use [Visual Studio Code](https://code.visualstudio.com/) to develop and
 
 VSCode has good support for debugging JavaScript, but to work with C++ code, you are required to install two additional VSCode extensions:
 
-* Microsoft C/C++
-* CodeLLDB
+- Microsoft C/C++
+- CodeLLDB
 
 To begin, you will need to build the Node addon and prepare the test environment:
+
 ```sh
 npm install --build-from-source --debug
 (cd tests && npm install)
 ```
 
-Prior to begin debugging, you must start Realm Object Server. In VSCode, under menu *Tasks*/*Run Task*, find *Download and Start Server*.
+Prior to begin debugging, you must start Realm Object Server. In VSCode, under menu _Tasks_/_Run Task_, find _Download and Start Server_.
+
+In the debugging pane, you can find `Debug LLDB + Node.js` in the dropdown. First select _Start Debugging_ in the _Debug_ menu.
+
+### Debugging failing Github Actions CI tests
 
-In the debugging pane, you can find `Debug LLDB + Node.js` in the dropdown. First select *Start Debugging* in the *Debug* menu.
+To debug failing Github Actions CI tests, it can be helpful to `ssh` into the runner and test out the CI commands manually. This Github Action can be used to add a step into the workflow which pauses it and outputs details to `ssh` into it: https://github.com/marketplace/actions/debugging-with-tmate
+
+The relevant snippet is:
+
+```
+- name: Setup tmate session
+  uses: mxschmitt/action-tmate@v3
+  with:
+      limit-access-to-actor: true
+  timeout-minutes: 30
+```
 
 ## Testing against real apps
 
 There are a couple of suggested workflows for testing your changes to Realm JS against real apps:
 
-* [Guide: Setting up watchman to copy changes from this package to an app](guide-watchman.md)
-* [Guide: Testing your changes against sample apps using a script](guide-testing-with-sample-apps.md)
+- [Guide: Setting up watchman to copy changes from this package to an app](guide-watchman.md)
+- [Guide: Testing your changes against sample apps using a script](guide-testing-with-sample-apps.md)