Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce 'decodeEnumsCaseInsensitive' feature to Json. #2345

Merged
merged 4 commits into from
Jul 5, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 50 additions & 15 deletions docs/json.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ In this chapter, we'll walk through features of [JSON](https://www.json.org/json
* [Allowing structured map keys](#allowing-structured-map-keys)
* [Allowing special floating-point values](#allowing-special-floating-point-values)
* [Class discriminator for polymorphism](#class-discriminator-for-polymorphism)
* [Decoding enums in a case-insensitive manner](#decoding-enums-in-a-case-insensitive-manner)
* [Global naming strategy](#global-naming-strategy)
* [Json elements](#json-elements)
* [Parsing to Json element](#parsing-to-json-element)
Expand Down Expand Up @@ -469,6 +470,39 @@ As you can see, discriminator from the `Base` class is used:

<!--- TEST -->

### Decoding enums in a case-insensitive manner

[Kotlin's naming policy recommends](https://kotlinlang.org/docs/coding-conventions.html#property-names) naming enum values
using either uppercase underscore-separated names or upper camel case names.
[Json] uses exact Kotlin enum values names for decoding by default.
However, sometimes third-party JSONs have such values named in lowercase or some mixed case.
In this case, it is possible to decode enum values in a case-insensitive manner using [JsonBuilder.decodeEnumsCaseInsensitive] property:

```kotlin
val format = Json { decodeEnumsCaseInsensitive = true }

enum class Cases { VALUE_A, @JsonNames("Alternative") VALUE_B }

@Serializable
data class CasesList(val cases: List<Cases>)

fun main() {
println(format.decodeFromString<CasesList>("""{"cases":["value_A", "alternative"]}"""))
}
```

> You can get the full code [here](../guide/example/example-json-12.kt).

It affects serial names as well as alternative names specified with [JsonNames] annotation, so both values are successfully decoded:

```text
CasesList(cases=[VALUE_A, VALUE_B])
```

This property does not affect encoding in any way.

<!--- TEST -->

### Global naming strategy

If properties' names in Json input are different from Kotlin ones, it is recommended to specify the name
Expand All @@ -489,7 +523,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-12.kt).
> You can get the full code [here](../guide/example/example-json-13.kt).

As you can see, both serialization and deserialization work as if all serial names are transformed from camel case to snake case:

Expand Down Expand Up @@ -541,7 +575,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-13.kt).
> You can get the full code [here](../guide/example/example-json-14.kt).

A `JsonElement` prints itself as a valid JSON:

Expand Down Expand Up @@ -584,7 +618,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-14.kt).
> You can get the full code [here](../guide/example/example-json-15.kt).

The above example sums `votes` in all objects in the `forks` array, ignoring the objects that have no `votes`:

Expand Down Expand Up @@ -624,7 +658,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-15.kt).
> You can get the full code [here](../guide/example/example-json-16.kt).

As a result, you get a proper JSON string:

Expand Down Expand Up @@ -653,7 +687,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-16.kt).
> You can get the full code [here](../guide/example/example-json-17.kt).

The result is exactly what you would expect:

Expand Down Expand Up @@ -699,7 +733,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-17.kt).
> You can get the full code [here](../guide/example/example-json-18.kt).

Even though `pi` was defined as a number with 30 decimal places, the resulting JSON does not reflect this.
The [Double] value is truncated to 15 decimal places, and the String is wrapped in quotes - which is not a JSON number.
Expand Down Expand Up @@ -739,7 +773,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-18.kt).
> You can get the full code [here](../guide/example/example-json-19.kt).

`pi_literal` now accurately matches the value defined.

Expand Down Expand Up @@ -779,7 +813,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-19.kt).
> You can get the full code [here](../guide/example/example-json-20.kt).

The exact value of `pi` is decoded, with all 30 decimal places of precision that were in the source JSON.

Expand All @@ -801,7 +835,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-20.kt).
> You can get the full code [here](../guide/example/example-json-21.kt).

```text
Exception in thread "main" kotlinx.serialization.json.internal.JsonEncodingException: Creating a literal unquoted value of 'null' is forbidden. If you want to create JSON null literal, use JsonNull object, otherwise, use JsonPrimitive
Expand Down Expand Up @@ -877,7 +911,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-21.kt).
> You can get the full code [here](../guide/example/example-json-22.kt).

The output shows that both cases are correctly deserialized into a Kotlin [List].

Expand Down Expand Up @@ -929,7 +963,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-22.kt).
> You can get the full code [here](../guide/example/example-json-23.kt).

You end up with a single JSON object, not an array with one element:

Expand Down Expand Up @@ -974,7 +1008,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-23.kt).
> You can get the full code [here](../guide/example/example-json-24.kt).

See the effect of the custom serializer:

Expand Down Expand Up @@ -1047,7 +1081,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-24.kt).
> You can get the full code [here](../guide/example/example-json-25.kt).

No class discriminator is added in the JSON output:

Expand Down Expand Up @@ -1143,7 +1177,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-25.kt).
> You can get the full code [here](../guide/example/example-json-26.kt).

This gives you fine-grained control on the representation of the `Response` class in the JSON output:

Expand Down Expand Up @@ -1208,7 +1242,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-26.kt).
> You can get the full code [here](../guide/example/example-json-27.kt).

```text
UnknownProject(name=example, details={"type":"unknown","maintainer":"Unknown","license":"Apache 2.0"})
Expand Down Expand Up @@ -1262,6 +1296,7 @@ The next chapter covers [Alternative and custom formats (experimental)](formats.
[JsonBuilder.allowSpecialFloatingPointValues]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-builder/allow-special-floating-point-values.html
[JsonBuilder.classDiscriminator]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-builder/class-discriminator.html
[JsonClassDiscriminator]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-class-discriminator/index.html
[JsonBuilder.decodeEnumsCaseInsensitive]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-builder/decode-enums-case-insensitive.html
[JsonBuilder.namingStrategy]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-builder/naming-strategy.html
[JsonNamingStrategy]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-naming-strategy/index.html
[JsonElement]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-element/index.html
Expand Down
1 change: 1 addition & 0 deletions docs/serialization-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ Once the project is set up, we can start serializing some classes.
* <a name='allowing-structured-map-keys'></a>[Allowing structured map keys](json.md#allowing-structured-map-keys)
* <a name='allowing-special-floating-point-values'></a>[Allowing special floating-point values](json.md#allowing-special-floating-point-values)
* <a name='class-discriminator-for-polymorphism'></a>[Class discriminator for polymorphism](json.md#class-discriminator-for-polymorphism)
* <a name='decoding-enums-in-a-case-insensitive-manner'></a>[Decoding enums in a case-insensitive manner](json.md#decoding-enums-in-a-case-insensitive-manner)
* <a name='global-naming-strategy'></a>[Global naming strategy](json.md#global-naming-strategy)
* <a name='json-elements'></a>[Json elements](json.md#json-elements)
* <a name='parsing-to-json-element'></a>[Parsing to Json element](json.md#parsing-to-json-element)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
package kotlinx.serialization.features

import kotlinx.serialization.*
import kotlinx.serialization.json.*
import kotlinx.serialization.test.*
import kotlin.test.*

@Suppress("EnumEntryName")
class JsonEnumsCaseInsensitiveTest: JsonTestBase() {
@Serializable
data class Foo(
val one: Bar = Bar.BAZ,
val two: Bar = Bar.QUX,
val three: Bar = Bar.QUX
)

enum class Bar { BAZ, QUX }

// It seems that we no longer report a warning that @Serializable is required for enums with @SerialName.
// It is still required for them to work at top-level.
sandwwraith marked this conversation as resolved.
Show resolved Hide resolved
@Serializable
enum class Cases {
ALL_CAPS,
MiXed,
all_lower,

@JsonNames("AltName")
hasAltNames,

@SerialName("SERIAL_NAME")
hasSerialName
}

@Serializable
data class EnumCases(val cases: List<Cases>)

val json = Json(default) { decodeEnumsCaseInsensitive = true }

@Test
fun testCases() = noLegacyJs { parametrizedTest { mode ->
val input =
"""{"cases":["ALL_CAPS","all_caps","mixed","MIXED","miXed","all_lower","ALL_LOWER","all_Lower","hasAltNames","HASALTNAMES","altname","ALTNAME","AltName","SERIAL_NAME","serial_name"]}"""
val target = listOf(
Cases.ALL_CAPS,
Cases.ALL_CAPS,
Cases.MiXed,
Cases.MiXed,
Cases.MiXed,
Cases.all_lower,
Cases.all_lower,
Cases.all_lower,
Cases.hasAltNames,
Cases.hasAltNames,
Cases.hasAltNames,
Cases.hasAltNames,
Cases.hasAltNames,
Cases.hasSerialName,
Cases.hasSerialName
)
val decoded = json.decodeFromString<EnumCases>(input, mode)
assertEquals(EnumCases(target), decoded)
val encoded = json.encodeToString(decoded, mode)
assertEquals(
"""{"cases":["ALL_CAPS","ALL_CAPS","MiXed","MiXed","MiXed","all_lower","all_lower","all_lower","hasAltNames","hasAltNames","hasAltNames","hasAltNames","hasAltNames","SERIAL_NAME","SERIAL_NAME"]}""",
encoded
)
}}

@Test
fun testTopLevelList() = noLegacyJs { parametrizedTest { mode ->
val input = """["all_caps","serial_name"]"""
val decoded = json.decodeFromString<List<Cases>>(input, mode)
assertEquals(listOf(Cases.ALL_CAPS, Cases.hasSerialName), decoded)
assertEquals("""["ALL_CAPS","SERIAL_NAME"]""", json.encodeToString(decoded, mode))
}}

@Test
fun testTopLevelEnum() = noLegacyJs { parametrizedTest { mode ->
val input = """"altName""""
val decoded = json.decodeFromString<Cases>(input, mode)
assertEquals(Cases.hasAltNames, decoded)
assertEquals(""""hasAltNames"""", json.encodeToString(decoded, mode))
}}

@Test
fun testSimpleCase() = parametrizedTest { mode ->
val input = """{"one":"baz","two":"Qux","three":"QUX"}"""
val decoded = json.decodeFromString<Foo>(input, mode)
assertEquals(Foo(), decoded)
assertEquals("""{"one":"BAZ","two":"QUX","three":"QUX"}""", json.encodeToString(decoded, mode))
}

enum class E { VALUE_A, @JsonNames("ALTERNATIVE") VALUE_B }

@Test
fun testDocSample() = noLegacyJs {

val j = Json { decodeEnumsCaseInsensitive = true }
@Serializable
data class Outer(val enums: List<E>)

println(j.decodeFromString<Outer>("""{"enums":["value_A", "alternative"]}""").enums)
}

@Test
fun testCoercingStillWorks() = parametrizedTest { mode ->
sandwwraith marked this conversation as resolved.
Show resolved Hide resolved
val withCoercing = Json(json) { coerceInputValues = true }
val input = """{"one":"baz","two":"unknown","three":"Que"}"""
assertEquals(Foo(), withCoercing.decodeFromString<Foo>(input, mode))
}

@Test
fun testFeatureDisablesProperly() = parametrizedTest { mode ->
val disabled = Json(json) {
coerceInputValues = true
decodeEnumsCaseInsensitive = false
}
val input = """{"one":"BAZ","two":"BAz","three":"baz"}""" // two and three should be coerced to QUX
assertEquals(Foo(), disabled.decodeFromString<Foo>(input, mode))
}

@Serializable enum class BadEnum { Bad, BAD }
sandwwraith marked this conversation as resolved.
Show resolved Hide resolved

@Test
fun testLowercaseClashThrowsException() = parametrizedTest { mode ->
assertFailsWithMessage<SerializationException>("""The suggested name 'bad' for enum value BAD is already one of the names for enum value Bad""") {
// an explicit serializer is required for JSLegacy
json.decodeFromString(Box.serializer(BadEnum.serializer()),"""{"boxed":"bad"}""", mode)
}
assertFailsWithMessage<SerializationException>("""The suggested name 'bad' for enum value BAD is already one of the names for enum value Bad""") {
json.decodeFromString(Box.serializer(BadEnum.serializer()),"""{"boxed":"unrelated"}""", mode)
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ class JsonNamingStrategyTest : JsonTestBase() {

val jsonWithNaming = Json(default) {
namingStrategy = JsonNamingStrategy.SnakeCase
decodeEnumsCaseInsensitive = true // check that related feature does not break anything
}

@Test
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ inline fun assertFailsWithSerialMessage(
)
assertTrue(
exception.message!!.contains(message),
"expected:<${exception.message}> but was:<$message>"
"expected:<$message> but was:<${exception.message}>"
)
}
inline fun <reified T : Throwable> assertFailsWithMessage(
Expand All @@ -89,6 +89,6 @@ inline fun <reified T : Throwable> assertFailsWithMessage(
val exception = assertFailsWith(T::class, assertionMessage, block)
assertTrue(
exception.message!!.contains(message),
"expected:<${exception.message}> but was:<$message>"
"expected:<$message> but was:<${exception.message}>"
)
}
3 changes: 3 additions & 0 deletions formats/json/api/kotlinx-serialization-json.api
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ public final class kotlinx/serialization/json/JsonBuilder {
public final fun getAllowStructuredMapKeys ()Z
public final fun getClassDiscriminator ()Ljava/lang/String;
public final fun getCoerceInputValues ()Z
public final fun getDecodeEnumsCaseInsensitive ()Z
public final fun getEncodeDefaults ()Z
public final fun getExplicitNulls ()Z
public final fun getIgnoreUnknownKeys ()Z
Expand All @@ -102,6 +103,7 @@ public final class kotlinx/serialization/json/JsonBuilder {
public final fun setAllowStructuredMapKeys (Z)V
public final fun setClassDiscriminator (Ljava/lang/String;)V
public final fun setCoerceInputValues (Z)V
public final fun setDecodeEnumsCaseInsensitive (Z)V
public final fun setEncodeDefaults (Z)V
public final fun setExplicitNulls (Z)V
public final fun setIgnoreUnknownKeys (Z)V
Expand Down Expand Up @@ -129,6 +131,7 @@ public final class kotlinx/serialization/json/JsonConfiguration {
public final fun getAllowStructuredMapKeys ()Z
public final fun getClassDiscriminator ()Ljava/lang/String;
public final fun getCoerceInputValues ()Z
public final fun getDecodeEnumsCaseInsensitive ()Z
public final fun getEncodeDefaults ()Z
public final fun getExplicitNulls ()Z
public final fun getIgnoreUnknownKeys ()Z
Expand Down
Loading