Are there any options to tune kotlinx.serilization in regards to object allocations?

[u6f6o]

What is your use-case?
In the last few weeks I migrated one of our services from spring to quarkus. In the process, I also had a look on alternatives to jackson json serialization as there are still some issue when using jackson in a native image. Kotlinx.serialization looked very promising, specifically as most of the setup is done in during compile time. After integrating kotlinx.serilization, I did some load tests and the latencies looked quite nice.

Shortly afterwards, I deployed this service on our prod environment and realised that the garbage collection memory pressure was much higher than before. Given these circumstances, I repeated my load tests and compared the impact on garbage collections before and after kotlinx.serialization was integrated:

Before:

After:

Are there any possibility to tune kotlinx.serilization in regards to object allocations? Given the much higher garbage collection pressure, I had to stick with our current serializer for the time being.

[qwwdfsad]

Hi, we frequently do tune our performance and try to stay on par with existing solutions (though we do not talk about it much as advertising performance wins on synthetic benchmarks is rarely a good way to go) and I'm surprised to see such high memory pressure.

Could you please show the heavy hitters of your load?
The best way to do so will be to show a dump from async-profiler, either CPU or alloc (it's likely that it can be seen on both).

I would suspect some forgotten place like Enum.values() allocation or some non-trivial serializer (e.g. sealed-classes related) is not properly cached

[u6f6o]

Hi,

Yeah, I agree, synthetic tests tend to be a bit artificial. I noticed analogue numbers on our production systems as well though. So far I did not use any custom serializers and the like, just pure data classes and relying on the quarkus integration. There is one exception when parsing the body of a jwt. In this case, I refer to the serializer() attached to the data class. Here are two flame graphes (obj allocations) before and after:

[qwwdfsad]

Thanks, this is really helpful! This is indeed a IO stream integration inefficiency, we'll see what we can do here

[u6f6o]

Hi @qwwdfsad, Just wanted to ask if there is any progress on this topic?

[qwwdfsad]

yes, it should've been fixed in #1901

[u6f6o]

I gave it another try and executed the same load tests again. It looks much better now (kotlin 1.7.20)

jackson:

kotlinx.serialization:

I suppose we can close the issue?

[sandwwraith]

I think so. Thank you again for your investigation and useful data!

[qwwdfsad]

@u6f6o thanks!

Could you please share both of the flamegraphs again? Or a benchmark, if that is possible.
I'm glad it helped with the overall load and the reported problem is gone, but I would like to push it further and make kotlinx-serialization faster than or being on-par with jackson

[u6f6o]

@qwwdfsad
Sure happy to help 👍. I'll post the flame graphs as soon as I am ready (could take some days though).

[u6f6o]

@qwwdfsad: I attached a zip with the async-profiler alloc flamegraphs for kotlinx-serialization and jackson.

Flamegraphs.zip

[qwwdfsad]

Thanks!

[qwwdfsad]

We should cache char arrays in ReaderJsonLexer, they are definitely way too big for the unconditional allocations.
We'll improve it futher 👍

[u6f6o]

@qwwdfsad : I noticed that quite a few changes have been applied. Is there already a kotlinx.serialization version available that I could test and create benchmarks for?

[qwwdfsad]

Nope, it will arrive a bit later in 1.5.0-RC (around a week or two from now)

[u6f6o]

The latest load tests look really good. Great work 👍

Kotlinx.serialization

Jackson: