Flexible sync

[tomduncalf]

What, How & Why?

This PR implements flexible sync support.

Note: once this PR is merged, you will need to supply BAAS_AWS_ACCESS_KEY_ID and BAAS_AWS_SECRET_ACCESS_KEY env vars when starting the Docker image.

Example

// Open a Realm with flexible sync enabled
const realm = new Realm({
  schema: [PersonSchema, DogSchema],
  sync: { user, flexible: true }
});

// ------------------------------------------------

// Get the current set of subscriptions
const subs = realm.getSubscriptions();

console.log(subs)
// Subscriptions { }
console.log(subs.empty);
// true
console.log(subs.snapshot());
// []

// ------------------------------------------------

// Add a subscription - updates can only be performed inside an `update` callback,
// and mutating methods only exist on the `MutableSubscriptions` class
subs.update(mutableSubs => {
  console.log(mutableSubs);
  // MutableSubscriptions {}
  mutableSubs.add(realm.objects("Person"));
});

console.log(subs.empty);
// false
console.log(subs.snapshot());
// [Subscription]
console.log(subs.state);
// "pending" - indicates the server has not finished processing this 
// update so the data locally may be missing/incomplete

// ------------------------------------------------

// Wait for server synchronisation to be complete
await subs.waitForSynchronisation();

console.log(subs.state);
// "complete" - all initial data is synced and we are in steady state synchronisation. 
// Data we/others add/remove which matches the subscription will be synced.

// ------------------------------------------------

let filteredSub;

// Add several subscriptions in a batch - this is efficient for the server,
// as the client ORs the queries together before sending, resulting in a 
// single update on the server side (which is an expensive operation)
subs.update(mutableSubs => {
  // Add a filtered subscription, and keep a reference to it
  filteredSub = mutableSubs.add(realm.objects("Person").filtered("age > 20"));
  // Add a named subscriptions
  mutableSubs.add(realm.objects("Dog"), { name: "dogSubs" });
});

// ------------------------------------------------

// Find a subscription by name
console.log(subs.findByName("dogSubs"))
// Subscription

// Find a subscription by query and inspect some of its properties
const foundSub = subs.find(realm.objects("Person").filtered("age > 20"));
console.log(foundSub.queryString);
// "age > 20"
console.log(foundSub.objectType);
// "Person"
console.log(foundSub.name);
// null

// ------------------------------------------------

// Various ways of removing subscriptions 
subs.update(mutableSubs => {
  // Remove by query - returns true if a subscription was removed
  mutableSubs.remove(realm.objects("Person"));
  // Remove by name - returns true if a subscription was removed
  mutableSubs.removeByName("dogSubs");
  // Remove by reference - returns true if a subscription was removed
  mutableSubs.removeSubscription(filteredSub);
  // Remove by object type - returns count of removed subscriptions
  mutableSubs.removeByObjectType("Person");
  // Remove all subscriptions - returns count of removed subscriptions
  mutableSubs.removeAll();
});

☑️ ToDos

• This shouldn't be merged until https://github.com/10gen/baas/pull/5414 is merged as apps cannot be imported without a hack in the app importer right now (removed for this PR: fbb6360)
• Determine correct behaviour for a couple of xit'ed tests
• Add more integration tests for complex scenarios, different data types etc. (maybe reuse mixed.ts tests for data types?)
• 📝 Changelog entry
• 📝 Compatibility label is updated or copied from previous entry
• 🚦 Tests
• 📱 Check the React Native/other sample apps work if necessary
• 📝 Public documentation PR created or is not necessary
• 💥 Breaking label has been applied or is not necessary

If this PR adds or changes public API's:

• typescript definitions file is updated
• jsdoc files updated
• Chrome debug API is updated if API is available on React Native

[takameyer]

Great work! Just have a few things I picked out and a possible discussion over naming conventions.

[kraenhansen]

I'm submitting what I've managed to review today. There's still code that I haven't read yet.

[kraenhansen]

I'm wondering what the right behaviour is here.

Considering the case that Nikola mentioned where the app is killed before the flexible sync session has stabilised. In this case the user might add a call to realm.subscriptions.waitForSynchronization to the startup of their app? If that's true, it might be better to just resolve the promise if there's nothing to wait for? This way the user doesn't have to check if there's a subscription before awaiting this.

[tomduncalf]

Yeah it's an interesting point. I'll bring this up for discussion and see if we can unify on an approach to it. I think if this is the desired behaviour, ideally core would behave like that rather than having SDK specific logic