Filters not working after upgrading to 4.4.0

[gjermundgaraba]

After trying to upgrade from 4.2.8 (tested on 4.3.0, seems to work OK there) to 4.4.0 I am getting errors in my tests because the filters doesn't seem to work as they used to. Only change is the upgrade to my kmongo-coroutine version.

Is there something in these filters that is expected to break between 4.3.0 and 4.4.0?

val filter = and(
    or(
        MyClass::someId eq someId,
        MyClass::someOtherId eq someOtherId,
        MyClass::nestedObject / NestedClass::yetAnotherId eq yetAnotherId
    ),
        MyClass::nestedObject exists true
)
collection
    .find(clientSession, filter)
    .sort(descending(MyClass::lastModifiedDate))
    .toList()

For instance I am now getting back documents that clearly does not have the "MyClass::nestedObject".

[zigzago]

There is this feature introduced in 4.4.0 #242 - you can now choose to not persist the null properties during insertion - may be your issue is related to this feature ? Should be backward compatible.

Because the generated query seems to be ok in 4.4.0:

{
  "$and": [
    {
      "$or": [
        {
          "someId": "someId"
        },
        {
          "someOtherId": "someOtherId"
        },
        {
          "nestedObject.yetAnotherId": "yetAnotherId"
        }
      ]
    },
    {
      "nestedObject": {
        "$exists": true
      }
    }
  ]
}

[gjermundgaraba]

Hmm, I haven't really touched anything, so it might seem like it is not entirely backwards compatible. I have tried to screw around with the settings, but can't seem to get it back to the previous behavior (that I didn't know I was depending on :D)

[zigzago]

Does nestedObject exist in the database ?

[gjermundgaraba]

Oh, I see what has changed, all values were suddenly serialized to null. It didn't use to do that. How can I turn that off?

[zigzago]

ObjectMappingConfiguration.serializeNull = false at application startup should solve the issue

Do you use kmongo, kmongo-native or kmongo-serialization ? How do you persist the objects ?

[gjermundgaraba]

It didn't seem to work.

I am using kmongo-coroutine.

This is the setup for creating the mongo client:

fun setUpMongoClient(dbUri: String, vararg jacksonModules: Module): CoroutineClient {
    val connectionString = ConnectionString(dbUri)
    val settings: MongoClientSettings = MongoClientSettings.builder()
        .applyConnectionString(connectionString)
        .retryWrites(true)
        .build()
    KMongoConfiguration.bsonMapper.registerModules(*jacksonModules)
    KMongoConfiguration.bsonMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL)
    return KMongo.createClient(settings).coroutine
}

The persistence is pretty basic I think (except for being inside transactions most of the time):

private val collection = mongoClient.getDatabase("db-name").getCollection<MyClass>("myclass")

suspend fun insertMyClass(myClass: MyClass) {
    // Simplified a bit, but generally:
    collection.insertOne(clientSession, myClass)
}

[Lezzio]

We had a similar issue but we were using the "save" method which actually ended up executing a replace query to MongoDB. The replace query can't support the exclusion of non-null values. Make sure to explicitly use the insert one and add "ObjectMappingConfiguration.serializeNull = false". Hope it could help you

[zigzago]

@bjaanes this should work with 4.4.0 and above:

fun setUpMongoClient(dbUri: String, vararg jacksonModules: Module): CoroutineClient {
    //must be called before any KMongoConfiguration call
    ObjectMappingConfiguration.serializeNull = false
    val connectionString = ConnectionString(dbUri)
    val settings: MongoClientSettings = MongoClientSettings.builder()
        .applyConnectionString(connectionString)
        .retryWrites(true)
        .build()
   //do not use bsonMapper directly 
    jacksonModules.forEach { KMongoConfiguration.registerModule(it) }
    //this is handled by ObjectMappingConfiguration.serializeNull 
    //KMongoConfiguration.bsonMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL)
    return KMongo.createClient(settings).coroutine
}

[zigzago]

@Lezzio I do reproduce the bug with ObjectMappingConfiguration.serializeNull = false & save/replace usage. I'm going to release a fix (see #319)