Optional JsonNull cannot be deserialized (always null, never JsonNull)

[pschichtel]

I have a class like this:

@Serializable
data class Property(val name: String, val defaultValue: JsonElement? = null)

The intent is: The default value should be an optional field, however if the field is given with null in JSON, then I want it decoded as JsonNull. That way I could differentiate between "no default value given" and "default value is null".

kotlinx.serialization currently always produces null in this case if the defaultValue field is nullable. If it is not nullable, then I correctly get JsonNull.

An ugly workaround (because it is an incompatible change) would be to have a wrapper object, that has a non-nullable field:

@Serializable
data class DefaultValue(val value: JsonElement)

@Serializable
data class Property(val name: String, val defaultValue: DefaultValue? = null)

Another ugly workaround, which at least isn't a breaking change, would be to make the defaultValue non-nullable with a dummy default value that signifies that the value wasn't given in JSON:

val NoDefault = JsonPrimitive(UUID.randomUUID().toString())

@Serializable
data class Property(val name: String, val defaultValue: JsonElement = NoDefault)

To Reproduce

I guess the code to reproduce this is fairly clear:

@Serializable
data class Property(val name: String, val defaultValue: JsonElement? = null)

fun main() {
    val test1 = Json.decodeFromString<Property>(
        """
            {
              "name": "Test1",
              "defaultValue": null
            }
        """.trimIndent()
    )
    val test2 = Json.decodeFromString<Property>(
        """
            {
              "name": "Test2"
            }
        """.trimIndent()
    )
    val test3 = Json.decodeFromJsonElement<Property>(
        JsonObject(
            mapOf(
                "name" to JsonPrimitive("Test3"),
                "defaultValue" to JsonNull
            )
        )
    )
    val test4 = Json.decodeFromJsonElement<Property>(
        JsonObject(
            mapOf(
                "name" to JsonPrimitive("Test4"),
            )
        )
    )

    println("${test1.name} -> ${test1.defaultValue == JsonNull}")
    println("${test2.name} -> ${test2.defaultValue == null}")
    println("${test3.name} -> ${test3.defaultValue == JsonNull}")
    println("${test4.name} -> ${test4.defaultValue == null}")
}

This produces

Test1 -> false
Test2 -> true
Test3 -> false
Test4 -> true

Expected behavior

Test1 -> true
Test2 -> true
Test3 -> true
Test4 -> true

Environment

• Kotlin version: 1.9.10
• Library version: 1.6.0
• Kotlin platforms: JVM
• Gradle version: 8.3

[pschichtel]

I decompiled the generated serializer and wrote a version of it that works:

@Serializable(PropertySerializer::class)
data class Property(val name: String, val defaultValue: JsonElement? = null)


object PropertySerializer : KSerializer<Property> {
    private val jsonElementSerializer = JsonElement.serializer()

    override val descriptor: SerialDescriptor = buildClassSerialDescriptor(Property::class.qualifiedName!!) {
        element("name", String.serializer().descriptor)
        element("defaultValue", jsonElementSerializer.descriptor)
    }

    override fun serialize(encoder: Encoder, value: Property) {
        TODO("Not yet implemented")
    }

    override fun deserialize(decoder: Decoder): Property {
        return (decoder as JsonDecoder).decodeStructure(descriptor) {
            var parsing = true
            var name: String? = null
            var defaultValue: JsonElement? = null

            while (parsing) {
                when (val index = decodeElementIndex(descriptor)) {
                    CompositeDecoder.DECODE_DONE ->
                        parsing = false
                    CompositeDecoder.UNKNOWN_NAME ->
                        throw SerializationException("unknown field!")
                    0 ->
                        name = decodeStringElement(descriptor, index)
                    1 ->
                        defaultValue = decodeSerializableElement(descriptor, index, jsonElementSerializer)
                }
            }
            if (name == null) {
                throw SerializationException("no name!")
            }
            Property(name, defaultValue)
        }
    }
}

[pschichtel]

I guess the root cause is, that the JsonElement and JsonPrimitive serializers' descriptors are not nullable.