Embedded Objects. Impossible to prevent "multiple backlinks" during migration

[sipersso]

Goals

I want to convert a RealmObject to an Embedded Object

Before Migration:

public class ChildObject:Object {}
public class ParentObject:Object {
    var children = List<ChildObject>()
}

public class AnotherObject:Object {
   var relatedChild:ChildObject?
}

Desired structure after migration:

public class ChildObject:EmbeddedObject{}
public class ParentObject:Object {
    var children = List<ChildObject>()
}

public class AnotherObject:Object {
     //No reference to related child
}

I want to convert the Child object to be embedded in it's parent, and remove the reference to the child object in the "AnotherObject" class

Expected Results

If the reference to a child is removed from AnotherObject, the there should only be one backlink left after migration.

Actual Results

The migration throws the following exception :

Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=io.realm Code=1 "At least one object does have multiple backlinks." UserInfo={NSLocalizedDescription=At least one object does have multiple backlinks., Error Code=1}

There doesn't seem to be a way to make sure that the Another object class removes the reference to Child and clears it's data. This is possible to do in Realm Java, but the migration API is very different for swift.

Here is what I have tried
1: Simply remove the property. This result in the "At least one object does have multiple backlinks". Setting the properties to nil does not seem to work either.
2: Set the anotherObject!["relatedChild"] = nil during migration. This leads to an "invalid property name child for class AnotherObject"

How can I make sure that the AnotherObject class is cleared of all the child references so that the migration can complete without problems?

Version of Realm and Tooling

Realm framework version: 10.7.4

Xcode version: 12.5
Dependency manager + version: SPM

Just to keep a timeline of this issue

• Issues is opened to be able to migrate Object to EmbeddedObject by nikola in name of the user Unable to execute a migration that changes the "embeddedness" of an object #7060, which comes from the forum on this issue https://developer.mongodb.com/community/forums/t/migrating-to-using-the-new-embedded-object-in-mongodb-realm/11920
• Issue fixed by this merge Fix backlink check in migration. realm-core#4414
• Then he is chatting on this issue Helper function for deleting orphaned embedded objects within migration #7145 about migrations function and his specific use case
• Last, he opened this issue to keep track of his specific issue, Embedded Objects. Impossible to prevent "multiple backlinks" during migration #7248
  Conclusion: This is the only user to ask for this feature and hasn't been able to solve his migration issues, we should continue all interaction on Embedded Objects. Impossible to prevent "multiple backlinks" during migration #7248

[DominicFrei]

@sipersso Using the example you created in #7145 I'd like to add the important part here as well:

About the more than one parent part: The same way you check if there is a child with an id that cannot be found in your parentMap you could check if there are multiple entries in the parentMap with the same childId. These need to be handled (most likely by creating a new child with the same values).

Would this be an option here?

[sipersso]

I don’t see how detecting the objects that have multiple links help if there is no way to delete the data?.

The property that I want to delete doesn’t exist after migration, so I don’t understand why the exception is thrown. Shouldn’t the data be deleted if I delete a property? It feels strange that if I remove the property it doesn’t get rid of the link and if I set it to nil it complains it doesn’t exist? Sounds like a bug?

There doesn’t seem to be a way to clear the data of a property during migration in the Swift API. The same migration works without problems in the Java SDK.

I want to delete the relatedChild references from AnotherObject altogether. If I do this, there will only be one backlink

[DominicFrei]

Some notes on that:

1. The problem is that removing the relatedChild property from AnotherObject does not automatically lead to every ChildObject only having one backlink.
   You need to make sure that every ChildObject only gets references by one of the children lists in ParentObject.
   If a ChildObject is references by multiple ParentObjects you will still get this error. Also, if the same ChildObject is added multiple times to the same list.

2. While in migration, the old (non-embedded) class still exist. If you want to look deeper into this: https://github.com/realm/realm-core/blob/master/src/realm/object-store/object_store.cpp#L834
   The change to embedded is applied after your migration block is executed.

3. About Java SDK: There are in fact - unfortunately - differences between the SDKs. Features that exist in one SDK will not necessarily exist / work the same in another SDK. Goal here would be to figure out if what you are doing in Java is actually not possible in Swift or if it is just not obvious how to do it and we need to improve our documentation on that.

[sipersso]

@DominicFrei Right.... I actually think that it was the error message that might have caused some confusion here. The problem wasn't that the child object had "multiple backlinks", the problem was that there was an orphaned child. If you look at the sample project I provided, I had missed doing this check for one of the object types. When adding the same type of check as I provided in the example here #7145 (comment) I was able to get the sample project I sent to you to complete the migration.

But the error message is misleading. If you get the message "At least one object does have multiple backlinks." you don't think that the root cause of this is that there is an orphaned child (no backlings). Maybe it would be great to change the error message to actually say that an orphan was detected. I believe the Android SDK does this if I am not mistaken. I'll make the change to the production code to see if this helps.

[DominicFrei]

@sipersso Can you provide an example for that?

The error message mentioned by you can only occur in case there are multiple backlinks:
https://github.com/realm/realm-core/blob/master/src/realm/table.cpp#L1063

If you are able to reproduce this problem where there are no multiple backlinks this would be a bug. My guess is there are orphaned objects that you see but also at least one with multiple links.

If the backlink count was 0 (orphaned) this would be the error message:
https://github.com/realm/realm-core/blob/master/src/realm/table.cpp#L1060

[sipersso]

Yes, I am able to reproduce it and have submitted a sample project based in support case 00771573 on Atlas support that reproduces the issue. The sample project should have been forwarded to you? If not, where can I send it?

I have both an example that has the problem and another sample where removing the orphaned object fixed the problem. I am not able to publish the sample here since it is based on my production data.

[DominicFrei]

Thanks you, @sipersso. I have received the example in the meantime. We'll look into it.

[sipersso]

@DominicFrei Maybe it is a combination that causes the error message? I did not only change an object to embedded, but I also removed a property that would have caused the "At least one object has multiple backlinks". Maybe the orphaned objects don't get changed in the same way as those that are now being embedded? I hope the sample helps you find the issue ;)

[sipersso]

I am sorry @DominicFrei, I was mistaken in that I had fixed the migration. The second sample I sent doesn't solve the problem, it justs deletes all Child objects (and then it is no wonder that migration works). Just proves how tricky it is to write these migration functions ;) I'll continue to investigate, but you can ignore the second sample in the 00771573 support case. It doesn't fix the problem.

[DominicFrei]

Thanks for letting us know. @sipersso
I have adjusted your example a bit to look into the objects further. The error would indeed only show up if there were multiple backlinks for the same objects. But there are not (when inspecting the Realm file directly).
@dianaafanador3 and I are investigating further into the possibility of the way we count backlinks being wrong.

Anyways, I agree with you, it's not obvious what's going on here and how a user is supposed to handle this migration. 😄
Depending on the root problem we either need a bug fix or update the documentation. 💪

[sipersso]

1. The problem is that removing the relatedChild property from AnotherObject does not automatically lead to every ChildObject only having one backlink.

The problem is that this isn't possible to do if you remove the property. If the relatedChild property is removed, there is no api for clearing the data.

The only workaround that I can think of is to keep the reference to "AnotherObject" for the only reason that this makes it possible clear the backlink from Child to AnotherObject.

public class ChildObject:EmbeddedObject{}
public class ParentObject:Object {
    var children = List<ChildObject>()
}

public class AnotherObject:Object {
    private var relatedChild:ChildObject? //Keep this reference only to be able to set it to nil during migration
}

I just tried this and then the migration works without problems, but then I am left with an unused relatedChild property in the AnotherObject class for both the iOS and Android app, which isn't that nice.

Shouldn't it be possible to clear the backlink if I remove the property? Seems strange that there isn't a way to do this.

[dianaafanador3]

Hi @sipersso I see that you opened another issue after successfully doing the migration. I know that migration finally worked after the above workaround.
We are aware that there is some fixing in core that need to be take care of and some helper function for this type of migrations, so we are taking some notes on that.
I'm closing this issue and focus on the new issue with sync.

[sipersso]

The workaround isn’t great. I will end up with an unwanted property in the sync schema that I will never be able to remove since I am not allowed to do destructive schema changes.

[DominicFrei]

Good news, @ironage figured this one out. 💪
There was indeed a problem where "At least one object does have multiple backlinks." could be shown even though "At least one object does not have a backlink (data would get lost)." would have been correct. This is not due to incorrect counting but due to the order in which things happen within a migration.
Fixed with realm/realm-core#4694 (needs to be released in Core and Cocoa).

[sipersso]

Awesome! Great job @ironage