ClassCastException when reading (decrypting) using @ExplicitEncrypted on custom type instead of simple type

[christianblust]

Hi all,

I am currently trying to implement explicit client side field level encryption according to https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongo.encryption.explicit

It works when I annotate simple types like String. However if I annotate a custom "Address" like type encryption works and my object is saved encrypted to mongo db. Although trying to read it results in a ClassCastException in the MongoEncryptionConverter:

java.lang.ClassCastException: class org.bson.Document cannot be cast to class org.bson.BsonValue (org.bson.Document and org.bson.BsonValue are in unnamed module of loader 'app')
	at org.springframework.data.mongodb.core.convert.encryption.MongoEncryptionConverter.lambda$decrypt$1(MongoEncryptionConverter.java:102)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.springframework.data.mongodb.core.convert.encryption.MongoEncryptionConverter.decrypt(MongoEncryptionConverter.java:101)
	at org.springframework.data.mongodb.core.convert.encryption.EncryptingConverter.read(EncryptingConverter.java:32)
	at org.springframework.data.mongodb.core.convert.encryption.MongoEncryptionConverter.read(MongoEncryptionConverter.java:65)
	at org.springframework.data.mongodb.core.convert.encryption.MongoEncryptionConverter.read(MongoEncryptionConverter.java:48)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter$MongoDbPropertyValueProvider.getPropertyValue(MappingMongoConverter.java:1920)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter$AssociationAwareMongoDbPropertyValueProvider.getPropertyValue(MappingMongoConverter.java:1983)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter$AssociationAwareMongoDbPropertyValueProvider.getPropertyValue(MappingMongoConverter.java:1944)
	at org.springframework.data.mapping.model.PersistentEntityParameterValueProvider.getParameterValue(PersistentEntityParameterValueProvider.java:71)
	at org.springframework.data.mapping.model.SpELExpressionParameterValueProvider.getParameterValue(SpELExpressionParameterValueProvider.java:49)
	at org.springframework.data.mapping.model.KotlinClassGeneratingEntityInstantiator$DefaultingKotlinClassInstantiatorAdapter.extractInvocationArguments(KotlinClassGeneratingEntityInstantiator.java:222)
	at org.springframework.data.mapping.model.KotlinClassGeneratingEntityInstantiator$DefaultingKotlinClassInstantiatorAdapter.createInstance(KotlinClassGeneratingEntityInstantiator.java:196)
	at org.springframework.data.mapping.model.ClassGeneratingEntityInstantiator.createInstance(ClassGeneratingEntityInstantiator.java:98)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:491)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.readDocument(MappingMongoConverter.java:459)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:395)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:391)
	at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:102)
	at org.springframework.data.mongodb.core.MongoTemplate$ReadDocumentCallback.doWith(MongoTemplate.java:3217)
	at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:2851)
	at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2556)
	at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2537)
	at org.springframework.data.mongodb.core.MongoTemplate.find(MongoTemplate.java:865)
	at org.springframework.data.mongodb.repository.support.SimpleMongoRepository.findAll(SimpleMongoRepository.java:354)
	at org.springframework.data.mongodb.repository.support.SimpleMongoRepository.findAll(SimpleMongoRepository.java:136)
	at jdk.internal.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.data.repository.core.support.RepositoryMethodInvoker$RepositoryFragmentMethodInvoker.lambda$new$0(RepositoryMethodInvoker.java:288)
	at org.springframework.data.repository.core.support.RepositoryMethodInvoker.doInvoke(RepositoryMethodInvoker.java:136)
	at org.springframework.data.repository.core.support.RepositoryMethodInvoker.invoke(RepositoryMethodInvoker.java:120)
	at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:516)
	at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:285)
	at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:628)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:168)
	at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:143)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:72)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.data.repository.core.support.MethodInvocationValidator.invoke(MethodInvocationValidator.java:94)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:244)
	at jdk.proxy3/jdk.proxy3.$Proxy154.findAll(Unknown Source)
	at jdk.internal.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:244)
	at jdk.proxy3/jdk.proxy3.$Proxy154.findAll(Unknown Source)

Probably relevant code parts:

@Configuration
class ClientEncryptionConfig(private val masterKeyProvider: MasterKeyProvider) {

    @Bean
    fun encryptionClient(): ClientEncryption {
        val clientEncryptionSettings = ClientEncryptionSettings.builder()
            .keyVaultMongoClientSettings(
                MongoClientSettings.builder()
                    .applyConnectionString(MongoConfig.connectionString)
                    .build()
            )
            .keyVaultNamespace(MongoConfig.keyVaultNamespace)
            .kmsProviders(masterKeyProvider.kmsProviders())
            .build()
        return ClientEncryptions.create(clientEncryptionSettings)
    }
}

@Configuration
class MongoConfig(private val clientEncryption: ClientEncryption, private val masterKeyProvider: MasterKeyProvider, private val appContext: ApplicationContext): AbstractMongoClientConfiguration(){

    companion object {
        val vaultCollectionName = "__keyVault"
        val dbName = "databaseName"
        val keyVaultNamespace = "$dbName.$vaultCollectionName"
        val connectionString = ConnectionString("connectionString")
    }

    @Bean
    override fun mongoClient(): MongoClient {
        val autoEncryptionSettings = AutoEncryptionSettings.builder()
            .keyVaultNamespace(keyVaultNamespace)
            .kmsProviders(masterKeyProvider.kmsProviders())
            .bypassAutoEncryption(true)
            .build()

        return MongoClients.create(
            MongoClientSettings.builder()
                .applyConnectionString(connectionString)
                .autoEncryptionSettings(autoEncryptionSettings)
                .build()
        )
    }

    @Bean
    fun encryptingConverter(): MongoEncryptionConverter{
        val encryption = MongoClientEncryption.just(clientEncryption)
        val keyResolver = EncryptionKeyResolver.annotated{ _ -> EncryptionKey.keyAltName("demo-data-key")}
        return MongoEncryptionConverter(encryption, keyResolver)
    }

    override fun configureConverters(converterConfigurationAdapter: MongoCustomConversions.MongoConverterConfigurationAdapter) {
        converterConfigurationAdapter.registerPropertyValueConverterFactory(PropertyValueConverterFactory.beanFactoryAware(appContext))
    }

    override fun getDatabaseName(): String {
        return "databaseName"
    }
}

Here reading (or decrypting) fails with the mentioned ClassCastException

@Document
data class TestDocument(
    val creationDate: Instant,
    @ExplicitEncrypted(algorithm = EncryptionAlgorithms.AEAD_AES_256_CBC_HMAC_SHA_512_Random, keyAltName = "demo-data-key") val addresses: List<AddressDocument>

This works:

@Document
data class AddressDocument(
    @ExplicitEncrypted(algorithm = EncryptionAlgorithms.AEAD_AES_256_CBC_HMAC_SHA_512_Random, keyAltName = "demo-data-key")  private val zip: String,
    @ExplicitEncrypted(algorithm = EncryptionAlgorithms.AEAD_AES_256_CBC_HMAC_SHA_512_Random, keyAltName = "demo-data-key") private val city: String) 

Sorry if you are also scanning stackoverflow, then this is probably redundant. However I am not sure if I stumbled upon a bug or if I just misconfigured something.

[christophstrobl]

It is hard to tell what's the problem here. The related test seems to work as expected. Please help us triage the issue and take the time to provide a complete minimal sample (something that we can unzip or git clone, build, and deploy) that reproduces the problem.

[christianblust]

Please start the docker-compose.yml (or any other mongocontainer) in /docker, then the Application.

The Startup Component saves a simple Address with @ExplicitEncrypted String fields and reads them successfully afterwards.

Then, a NestedDocument with an @ExplicitEncrypted List<NestedAddress> is saved and read. Reading results in the mentioned exception. When I change List<NestedAddress> to NestedAddress the error looks slightly different.

csfldebug.zip
https://github.com/christianblust/csfle-debug/tree/develop

[christophstrobl]

@christianblust thanks for the reproducer!

[christophstrobl]

In this case, since the driver is aware of the encryption configuration, the driver already decrypts the array field internally before it is even handed to the converter, which leads to the mentioned ClassCastException.

[christianblust]

Thanks for looking into it! I just noticed the same happening for java.time.Instant, but that happens probably for the same reason. Currently I am working around with a lot of @ExplicitEncrypted annotations and ignoring the Instants. Is that a viable interim solution until a fix is available?

[christophstrobl]

it is. you can try

@Bean
fun encryptingConverter(): MongoEncryptionConverter{
    val encryption = MongoClientEncryption.just(clientEncryption)
    val keyResolver = EncryptionKeyResolver.annotated{ _ -> EncryptionKey.keyAltName("demo-data-key")}
    return object : MongoEncryptionConverter(encryption, keyResolver)  {
        override fun decrypt(encryptedValue: Any, context: EncryptionContext): Any {
            return context.read(encryptedValue, context.property.typeInformation)
        }
    }
}

[christianblust]

This helps a bit, no exception! However, the result-type after reading or decrypting @ExplicitEncrypted(algorithm = EncryptionAlgorithms.AEAD_AES_256_CBC_HMAC_SHA_512_Random, keyAltName = "demo-data-key") private val nestedAddress: List<NestedAddress> = emptyList()) then is a Document, so no cast to NestedAddress is happening.

I also tested skipping the AutoEncryptionSettings in the MongoClient:

@Bean
override fun mongoClient(): MongoClient {
    //val autoEncryptionSettings = AutoEncryptionSettings.builder()
    //    .keyVaultNamespace(keyVaultNamespace)
    //    .kmsProviders(masterKeyProvider.kmsProviders())
    //    .bypassAutoEncryption(true)
    //    .build()

    return MongoClients.create(
        MongoClientSettings.builder()
            .applyConnectionString(connectionString)
            //.autoEncryptionSettings(autoEncryptionSettings)
            .build()
    )
}

So, @ExplicitEncrypted works on complex types as you already indicated. Does this solution have some major drawbacks? Code seems much cleaner for larger classes with that approach. java.time.Instant unfortunately still not works in that case.

Unable to convert 2023-07-04T12:56:09.668856Z (java.time.Instant) to BsonValue.