Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow encoding literal JSON values #2041

Merged
merged 31 commits into from
Oct 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
ff8fd99
implement JsonRawElement, for encoding literal JSON strings
aSemy Oct 13, 2022
ace3db0
update name of inlineRawJsonElementEncoder
aSemy Oct 13, 2022
ac474ed
Merge remote-tracking branch 'origin/dev' into feat/1051-big_decimals
aSemy Oct 13, 2022
92683c8
Merge remote-tracking branch 'origin/dev' into feat/1051-big_decimals
aSemy Oct 17, 2022
83043d4
rename JsonRawElement -> JsonUnquotedLiteral
aSemy Oct 17, 2022
462af90
finish renaming JsonRawElement -> JsonUnquotedLiteral
aSemy Oct 19, 2022
b79528d
update docs for JsonUnquotedLiteral
aSemy Oct 19, 2022
c437fc4
update docs for JsonUnquotedLiteral
aSemy Oct 19, 2022
28c318f
Merge remote-tracking branch 'origin/dev' into feat/1051-big_decimals
aSemy Oct 19, 2022
74a91a1
add knit docs for JsonUnquotedLiteral
aSemy Oct 19, 2022
816f50e
grammar/editing knit docs
aSemy Oct 19, 2022
e33fbe9
additional Knit test for BigDecimal decoding, and minor formatting/gr…
aSemy Oct 19, 2022
835816e
add 'opt-in' note
aSemy Oct 19, 2022
40e8f3f
add BigDecimal link
aSemy Oct 19, 2022
ef3d88e
minor grammar tweak
aSemy Oct 19, 2022
2309026
Update docs/json.md
aSemy Oct 20, 2022
e2fe1c4
Update docs/json.md
aSemy Oct 20, 2022
e430234
Update formats/json/commonMain/src/kotlinx/serialization/json/JsonEle…
aSemy Oct 20, 2022
c5e40dc
Update formats/json/commonMain/src/kotlinx/serialization/json/JsonEle…
aSemy Oct 20, 2022
374fcc0
Update formats/json/commonMain/src/kotlinx/serialization/json/JsonEle…
aSemy Oct 20, 2022
be8a5d0
Update formats/json/commonMain/src/kotlinx/serialization/json/JsonEle…
aSemy Oct 20, 2022
1374e85
move note regarding `"null"`
aSemy Oct 20, 2022
1a19d10
move non-parmeterized assert out of parametrizedTest {}
aSemy Oct 20, 2022
61a7c42
update 'knit'
aSemy Oct 20, 2022
d75ea07
bump knit version
aSemy Oct 20, 2022
5a8c304
rm unnecessary comma
aSemy Oct 20, 2022
de14566
fix link
aSemy Oct 20, 2022
a03fc10
fix TOC link (temp fix)
aSemy Oct 20, 2022
32f6d64
add note about creating a serializer
aSemy Oct 20, 2022
74a5e09
gramma fix, and fix link
aSemy Oct 20, 2022
4c2872b
fix link
aSemy Oct 20, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions core/api/kotlinx-serialization-core.api
Original file line number Diff line number Diff line change
Expand Up @@ -774,6 +774,10 @@ public final class kotlinx/serialization/internal/InlineClassDescriptor : kotlin
public fun isInline ()Z
}

public final class kotlinx/serialization/internal/InlineClassDescriptorKt {
public static final fun InlinePrimitiveDescriptor (Ljava/lang/String;Lkotlinx/serialization/KSerializer;)Lkotlinx/serialization/descriptors/SerialDescriptor;
}

public final class kotlinx/serialization/internal/IntArrayBuilder : kotlinx/serialization/internal/PrimitiveArrayBuilder {
public synthetic fun build$kotlinx_serialization_core ()Ljava/lang/Object;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,8 @@ internal class InlineClassDescriptor(
}
}

internal fun <T> InlinePrimitiveDescriptor(name: String, primitiveSerializer: KSerializer<T>): SerialDescriptor =
@InternalSerializationApi
public fun <T> InlinePrimitiveDescriptor(name: String, primitiveSerializer: KSerializer<T>): SerialDescriptor =
InlineClassDescriptor(name, object : GeneratedSerializer<T> {
// object needed only to pass childSerializers()
override fun childSerializers(): Array<KSerializer<*>> = arrayOf(primitiveSerializer)
Expand Down
168 changes: 161 additions & 7 deletions docs/json.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ In this chapter, we'll walk through features of [JSON](https://www.json.org/json
* [Types of Json elements](#types-of-json-elements)
* [Json element builders](#json-element-builders)
* [Decoding Json elements](#decoding-json-elements)
* [Encoding literal Json content (experimental)](#encoding-literal-json-content-experimental)
* [Serializing large decimal numbers](#serializing-large-decimal-numbers)
* [Using `JsonUnquotedLiteral` to create a literal unquoted value of `null` is forbidden](#using-jsonunquotedliteral-to-create-a-literal-unquoted-value-of-null-is-forbidden)
* [Json transformations](#json-transformations)
* [Array wrapping](#array-wrapping)
* [Array unwrapping](#array-unwrapping)
Expand Down Expand Up @@ -236,7 +239,7 @@ Project(name=kotlinx.serialization, language=Kotlin)
### Encoding defaults

Default values of properties are not encoded by default because they will be assigned to missing fields during decoding anyway.
See the [Defaults are not encoded](basic-serialization.md#defaults-are-not-encoded) section for details and an example.
See the [Defaults are not encoded](basic-serialization.md#defaults-are-not-encoded-by-default) section for details and an example.
This is especially useful for nullable properties with null defaults and avoids writing the corresponding null values.
The default behavior can be changed by setting the [encodeDefaults][JsonBuilder.encodeDefaults] property to `true`:

Expand Down Expand Up @@ -612,6 +615,153 @@ Project(name=kotlinx.serialization, language=Kotlin)

<!--- TEST -->

### Encoding literal Json content (experimental)

> This functionality is experimental and requires opting-in to [the experimental Kotlinx Serialization API](compatibility.md#experimental-api).

In some cases it might be necessary to encode an arbitrary unquoted value.
This can be achieved with [JsonUnquotedLiteral].

#### Serializing large decimal numbers

The JSON specification does not restrict the size or precision of numbers, however it is not possible to serialize
numbers of arbitrary size or precision using [JsonPrimitive()].

If [Double] is used, then the numbers are limited in precision, meaning that large numbers are truncated.
When using Kotlin/JVM [BigDecimal] can be used instead, but [JsonPrimitive()] will encode the value as a string, not a
number.

```kotlin
import java.math.BigDecimal

val format = Json { prettyPrint = true }

fun main() {
val pi = BigDecimal("3.141592653589793238462643383279")

val piJsonDouble = JsonPrimitive(pi.toDouble())
val piJsonString = JsonPrimitive(pi.toString())

val piObject = buildJsonObject {
put("pi_double", piJsonDouble)
put("pi_string", piJsonString)
}

println(format.encodeToString(piObject))
}
```

> You can get the full code [here](../guide/example/example-json-16.kt).

Even though `pi` was defined as a number with 30 decimal places, the resulting JSON does not reflect this.
The [Double] value is truncated to 15 decimal places, and the String is wrapped in quotes - which is not a JSON number.

```text
{
"pi_double": 3.141592653589793,
"pi_string": "3.141592653589793238462643383279"
}
```

<!--- TEST -->

To avoid precision loss, the string value of `pi` can be encoded using [JsonUnquotedLiteral].

```kotlin
import java.math.BigDecimal

val format = Json { prettyPrint = true }

fun main() {
val pi = BigDecimal("3.141592653589793238462643383279")

// use JsonUnquotedLiteral to encode raw JSON content
val piJsonLiteral = JsonUnquotedLiteral(pi.toString())

val piJsonDouble = JsonPrimitive(pi.toDouble())
val piJsonString = JsonPrimitive(pi.toString())

val piObject = buildJsonObject {
put("pi_literal", piJsonLiteral)
put("pi_double", piJsonDouble)
put("pi_string", piJsonString)
}

println(format.encodeToString(piObject))
}
```

> You can get the full code [here](../guide/example/example-json-17.kt).

`pi_literal` now accurately matches the value defined.

```text
{
"pi_literal": 3.141592653589793238462643383279,
"pi_double": 3.141592653589793,
"pi_string": "3.141592653589793238462643383279"
}
```

<!--- TEST -->

To decode `pi` back to a [BigDecimal], the string content of the [JsonPrimitive] can be used.

(This demonstration uses a [JsonPrimitive] for simplicity. For a more re-usable method of handling serialization, see
[Json Transformations](#json-transformations) below.)


```kotlin
import java.math.BigDecimal

fun main() {
val piObjectJson = """
{
"pi_literal": 3.141592653589793238462643383279
}
""".trimIndent()

val piObject: JsonObject = Json.decodeFromString(piObjectJson)

val piJsonLiteral = piObject["pi_literal"]!!.jsonPrimitive.content

val pi = BigDecimal(piJsonLiteral)

println(pi)
}
```

> You can get the full code [here](../guide/example/example-json-18.kt).

The exact value of `pi` is decoded, with all 30 decimal places of precision that were in the source JSON.

```text
3.141592653589793238462643383279
```

<!--- TEST -->

#### Using `JsonUnquotedLiteral` to create a literal unquoted value of `null` is forbidden

To avoid creating an inconsistent state, encoding a String equal to `"null"` is forbidden.
Use [JsonNull] or [JsonPrimitive] instead.

```kotlin
fun main() {
// caution: creating null with JsonUnquotedLiteral will cause an exception!
JsonUnquotedLiteral("null")
}
```

> You can get the full code [here](../guide/example/example-json-19.kt).

```text
Exception in thread "main" kotlinx.serialization.json.internal.JsonEncodingException: Creating a literal unquoted value of 'null' is forbidden. If you want to create JSON null literal, use JsonNull object, otherwise, use JsonPrimitive
```

<!--- TEST LINES_START -->


## Json transformations

To affect the shape and contents of JSON output after serialization, or adapt input to deserialization,
Expand Down Expand Up @@ -679,7 +829,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-16.kt).
> You can get the full code [here](../guide/example/example-json-20.kt).

The output shows that both cases are correctly deserialized into a Kotlin [List].

Expand Down Expand Up @@ -731,7 +881,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-17.kt).
> You can get the full code [here](../guide/example/example-json-21.kt).

You end up with a single JSON object, not an array with one element:

Expand Down Expand Up @@ -776,7 +926,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-18.kt).
> You can get the full code [here](../guide/example/example-json-22.kt).

See the effect of the custom serializer:

Expand Down Expand Up @@ -849,7 +999,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-19.kt).
> You can get the full code [here](../guide/example/example-json-23.kt).

No class discriminator is added in the JSON output:

Expand Down Expand Up @@ -945,7 +1095,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-20.kt).
> You can get the full code [here](../guide/example/example-json-24.kt).

This gives you fine-grained control on the representation of the `Response` class in the JSON output:

Expand Down Expand Up @@ -1010,7 +1160,7 @@ fun main() {
}
```

> You can get the full code [here](../guide/example/example-json-21.kt).
> You can get the full code [here](../guide/example/example-json-25.kt).

```text
UnknownProject(name=example, details={"type":"unknown","maintainer":"Unknown","license":"Apache 2.0"})
Expand All @@ -1025,8 +1175,10 @@ The next chapter covers [Alternative and custom formats (experimental)](formats.

<!-- references -->
[RFC-4627]: https://www.ietf.org/rfc/rfc4627.txt
[BigDecimal]: https://docs.oracle.com/javase/8/docs/api/java/math/BigDecimal.html

<!-- stdlib references -->
[Double]: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-double/
[Double.NaN]: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-double/-na-n.html
[List]: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/-list/
[Map]: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/-map/
Expand Down Expand Up @@ -1079,6 +1231,8 @@ The next chapter covers [Alternative and custom formats (experimental)](formats.
[buildJsonArray]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/build-json-array.html
[buildJsonObject]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/build-json-object.html
[Json.decodeFromJsonElement]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/decode-from-json-element.html
[JsonUnquotedLiteral]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-unquoted-literal.html
[JsonNull]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-null/index.html
[JsonTransformingSerializer]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-transforming-serializer/index.html
[Json.encodeToString]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json/encode-to-string.html
[JsonContentPolymorphicSerializer]: https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json-content-polymorphic-serializer/index.html
Expand Down
3 changes: 3 additions & 0 deletions docs/serialization-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,9 @@ Once the project is set up, we can start serializing some classes.
* <a name='types-of-json-elements'></a>[Types of Json elements](json.md#types-of-json-elements)
* <a name='json-element-builders'></a>[Json element builders](json.md#json-element-builders)
* <a name='decoding-json-elements'></a>[Decoding Json elements](json.md#decoding-json-elements)
* <a name='encoding-literal-json-content-experimental'></a>[Encoding literal Json content (experimental)](json.md#encoding-literal-json-content-experimental)
* <a name='serializing-large-decimal-numbers'></a>[Serializing large decimal numbers](json.md#serializing-large-decimal-numbers)
* <a name='using-jsonunquotedliteral-to-create-a-literal-unquoted-value-of-null-is-forbidden'></a>[Using `JsonUnquotedLiteral` to create a literal unquoted value of `null` is forbidden](json.md#using-jsonunquotedliteral-to-create-a-literal-unquoted-value-of-null-is-forbidden)
* <a name='json-transformations'></a>[Json transformations](json.md#json-transformations)
* <a name='array-wrapping'></a>[Array wrapping](json.md#array-wrapping)
* <a name='array-unwrapping'></a>[Array unwrapping](json.md#array-unwrapping)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,28 @@ class JsonPrimitiveSerializerTest : JsonTestBase() {
assertEquals(JsonPrimitiveWrapper(JsonPrimitive("239")), default.decodeFromString(JsonPrimitiveWrapper.serializer(), string, jsonTestingMode))
}

@Test
fun testJsonUnquotedLiteralNumbers() = parametrizedTest { jsonTestingMode ->
listOf(
"99999999999999999999999999999999999999999999999999999999999999999999999999",
"99999999999999999999999999999999999999.999999999999999999999999999999999999",
"-99999999999999999999999999999999999999999999999999999999999999999999999999",
"-99999999999999999999999999999999999999.999999999999999999999999999999999999",
"2.99792458e8",
"-2.99792458e8",
).forEach { literalNum ->
val literalNumJson = JsonUnquotedLiteral(literalNum)
val wrapper = JsonPrimitiveWrapper(literalNumJson)
val string = default.encodeToString(JsonPrimitiveWrapper.serializer(), wrapper, jsonTestingMode)
assertEquals("{\"primitive\":$literalNum}", string, "mode:$jsonTestingMode")
assertEquals(
JsonPrimitiveWrapper(literalNumJson),
default.decodeFromString(JsonPrimitiveWrapper.serializer(), string, jsonTestingMode),
"mode:$jsonTestingMode",
)
}
}

@Test
fun testTopLevelPrimitive() = parametrizedTest { jsonTestingMode ->
val string = default.encodeToString(JsonPrimitive.serializer(), JsonPrimitive(42), jsonTestingMode)
Expand Down Expand Up @@ -76,7 +98,7 @@ class JsonPrimitiveSerializerTest : JsonTestBase() {
}

@Test
fun testJsonLiterals() {
fun testJsonLiterals() {
testLiteral(0L, "0")
testLiteral(0, "0")
testLiteral(0.0, "0.0")
Expand Down
Loading