Stackoverflow error due to parentScope of Generics being pushed as the same object

[hierynomus]

In our product we're currently using Kryo 2.24.0 to serialize a data structure to another node. However in some specific cases during serialization a StackOverflowError occurs. with the following stacktrace:

java.lang.StackOverflowError
    at com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:61)
    at com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
    at com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
    at com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
    at com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
    at com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
    at com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
    at com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
    at com.esotericsoftware.kryo.Generics.getConcreteClass(Generics.java:62)
[...]

After much debugging we see that in this case kryo.pushGenericsScope(...) is called with exactly the same Generics instance as that is already set. When this happens, the parentScope points to the same instance as the current scope, so you get a loop while resolving.

I've been able to create an isolated testcase in the following Gist: https://gist.github.com/hierynomus/a2f143a555466f83d9bc

This issue is reproducible with Kryo 3.0.3 when using the Gradle buildfile (Gradle 2.6) that is attached to the Gist.

In order to fix this, my initial thought was returning a boolean from pushGenericsScope whether or not it was pushed, and checking before popping that there was actually a push. And then inside the pushGenericsScope checking whether the same instance was being pushed. However this fails the clirr checks.

You could also do the same check before calling the pushGenericsScope / popGenericsScope combo. This will adhere to the clirr checks.

I'd be happy to send a PR, but thought I'd discuss an approach first.

[hierynomus]

Small update (and further pinpointing)...

Running this on MacOSX with Java7 (1.7.0_71) gives a StackOverflowError
However running on Java8 (1.8.0_25) runs successfully.

So the real question is what is different when switching the Java version?

[hierynomus]

No, scratch that... The testcase contained a Set... Changing that to a List gives reproducible results. Gist updated.

[hierynomus]

PR was merged, so I think this issue can be closed.