Polymorphic deserialization doesn't deserialize base class fields

[devshorts]

I have a heirarchy like:

@JsonTypeInfo(
  use = JsonTypeInfo.Id.NAME,
  include = JsonTypeInfo.As.PROPERTY,
  property = "type",
  defaultImpl = classOf[DefaultMessage])
@JsonSubTypes(value = Array(
  new Type(value = classOf[ContinuationMessage], name = "v1:continuation")
))
sealed trait TestMessage extends BaseMessage
case class ContinuationMessage(sub1: String) extends TestMessage
case class DefaultMessage(sub: String) extends TestMessage

Where the base class that all sub classes inherit from looks like:

class BaseMessage {
   var base: Option[String] = None
}

When I deserialize json like below, which contains the fields of a subclass and the base class together:

{
  "sub": "foo", // subclass field 
  "base": "bar" // base class field
}

I get an instance of the class DefaultMessage properly since it is the default implementation, and I get the sub field of foo which is the field of the subclass, but I don't get the base of bar which is the field of the base class

If I deserialize directly to the DefaultMessage class instead of from the trait I get the field. So it seems that somehow the polymorphic serialization is not doing the base class?

Here is a full test:

@JsonTypeInfo(
  use = JsonTypeInfo.Id.NAME,
  include = JsonTypeInfo.As.PROPERTY,
  property = "type",
  defaultImpl = classOf[DefaultMessage])
@JsonSubTypes(value = Array(
  new Type(value = classOf[ContinuationMessage], name = "v1:continuation")
))
sealed trait TestM extends BaseMessage
case class ContinuationMessage(sub1: String) extends TestM
case class DefaultMessage(sub: String) extends TestM

class BaseMessage {
  var base: Option[String] = None
}

@RunWith(classOf[JUnitRunner])
class JsonTests extends FlatSpec with Matchers {
  "Base class fields" should "deserialize" in {
    val json: TestM = new JacksonSerializer().fromJson(
      """
        |{
        |  "sub": "foo",
        |  "base": "bar"
        |}
      """.
        stripMargin, classOf[TestM])

    json.base shouldBe Some("bar")
  }
}

[nbauernfeind]

What versions of jackson and the scala module are you using?

[devshorts]

@nbauernfeind I'm using jackson/scala module version 2.7.3 on scala-2.10.4

[devshorts]

@nbauernfeind any ideas? Can I help somehow? I'd love to get this resolved or at least have a workaround

[cowtowncoder]

@devshorts Make sure to try out newer versions; 2.7.4 had important fixes in type resolution. And jackson-databind 2.7.5 had one more fix I think so it may make sense to specify 2.7.5 of databind to see if that makes a difference beyond 2.7.4.

[devshorts]

Ok cool, I'll give it a shot tomorrow

[devshorts]

@cowtowncoder i tried 2.7.4 and used the 2.7.5 databind but still no dice. the base class fields are still not serialized into the type. Do you have any other suggestions, as this is becoming a big issue for my codebase

[cowtowncoder]

@devshorts Unfortunately I am not familiar with this module (or, Scala) so all I know is how Java side works. There are 2 main possiiblities here; either it is something general that jackson-databind is doing wrong, or something Scala module does (possibly by not using proper access methods from databind). At this point, given lack of known issues in this area in databind I am leaning towards latter since going 2.7.5 did not resolve the problem. 2.7.2, for example, did have specific issue with exclusion of super-type fields for polymorphic deserialization.

About the only suggestions I have is to make 100% sure versions are like you say if you haven't done that. With maven it'd be mvn dependency:tree. This because getting wrong version via transitive dependencies is easy and occasionally causes problems, esp. with patch versions.

You may also want to ask this on jackson-user mailing list just in case someone else knows more: not many users or devs outside of jackson core team follow issue updates.

[devshorts]

Ok thanks. Interestingly enough it works fine if the base trait is an abstract class and not a trait

[cowtowncoder]

That does seem weird. Not sure if it's basic class-vs-interface difference, or something with Scala type introspection.

[devshorts]

For completeness (in case someone finds this issue later), I did verify that I was using 2.7.5 databind and scala jackson 2.7.4 via the dependency tree.

And here is the jackson-users discussion: https://groups.google.com/forum/#!topic/jackson-user/f1G3E6U9W84

[cowtowncoder]

Actually I do have one more hypothesis: perhaps use of traits will add "invisible" intermediate classes between base class and implementation class, and this/these additional layer(s) could be the difference between abstract class approach. If so, perhaps it would be possible to reproduce the issue.

If so, the problem would very likely be within jackson-databind, and specifically in code that creates "specialized" types (that is, takes a resolved base type, type-erased subtype, and tries to resolve it with types bound in base type -- this is what happens for generic types during polymorphic deserialization).

Would it be easy enough to have a look at actual generated classes, to see how traits in this case translate to classes that JVM sees?

[devshorts]

@cowtowncoder sure can. Here is the decompiled (via jdgui) representation:

This is with traits

@JsonTypeInfo(use=JsonTypeInfo.Id.NAME, include=JsonTypeInfo.As.PROPERTY, property="type", defaultImpl=DefaultMessage.class)
@JsonSubTypes({@com.fasterxml.jackson.annotation.JsonSubTypes.Type(value=ContinuationMessage.class, name="v1:continuation")})
public abstract interface TestM {}

public class DefaultMessage
  extends BaseMessage
  implements TestM, Product, Serializable
{
  private final String sub;

  public DefaultMessage(String sub)
  {
    Product.class.$init$(this);
  }

...

public class BaseMessage
{
  public void base_$eq(Option<String> x$1)
  {
    this.base = x$1;
  }

  public Option<String> base()
  {
    return this.base;
  }

  private Option<String> base = None..MODULE$;
}

public class ContinuationMessage
  extends BaseMessage
  implements TestM, Product, Serializable
{
  private final String sub1;

  public ContinuationMessage(String sub1)
  {
    Product.class.$init$(this);
  }

...

This is with an abstract class

@JsonTypeInfo(use=JsonTypeInfo.Id.NAME, include=JsonTypeInfo.As.PROPERTY, property="type", defaultImpl=DefaultMessage.class)
@JsonSubTypes({@com.fasterxml.jackson.annotation.JsonSubTypes.Type(value=ContinuationMessage.class, name="v1:continuation")})
public abstract class TestM
  extends BaseMessage
{}

public class DefaultMessage
  extends TestM
  implements Product, Serializable
{
  private final String sub;

  public DefaultMessage(String sub)
  {
    Product.class.$init$(this);
  }
  ...

public class ContinuationMessage
  extends TestM
  implements Product, Serializable
{
  private final String sub1;

  public ContinuationMessage(String sub1)
  {
    Product.class.$init$(this);
  }
  ...

public class BaseMessage
{
  public void base_$eq(Option<String> x$1)
  {
    this.base = x$1;
  }

  public Option<String> base()
  {
    return this.base;
  }

  private Option<String> base = None..MODULE$;
}

I wonder if the issue is that when its a trait the subclasses are not bound to the tame type heirarchy with the base class. The base class is "mixed in" with each of the subclasses, so the interface itself doesn't know about those properties. But when its with an abstract class the serializer can tell that a TestM has the fields we expect

[devshorts]

If I make the BaseMessage a trait vs an abstract class then it does properly work because the bytecode creates:

public abstract interface TestM
  extends BaseMessage
{}