Serialization failure for fields after updating to Jackson 2.18.0

[arvgord]

Search before asking

• I searched in the issues and found nothing similar.
• I searched in the issues of databind and other modules used and found nothing similar.
• I have confirmed that the problem only occurs when using Kotlin.

Describe the bug

After upgrading Jackson from version 2.17.2 to 2.18.0, the serialization process for SecondObject and ThirdObject has completely broken. The fields are either missing or partially serialized, making the resulting JSON incomplete.

I expected that only the field order might change, but now fields that were present in the input JSON are missing from the serialized output entirely.

To Reproduce

Tested objects:

@JsonAutoDetect(
    fieldVisibility = JsonAutoDetect.Visibility.ANY,
    getterVisibility = JsonAutoDetect.Visibility.NONE
)
class FirstObject {
    @field:JsonAnySetter
    @field:JsonAnyGetter
    val data: Map<String, Any?> = LinkedHashMap()

    val transactionId: String
        get() = data["transactionId"] as String
}

@JsonAutoDetect(
    fieldVisibility = JsonAutoDetect.Visibility.ANY,
    getterVisibility = JsonAutoDetect.Visibility.NONE
)
class SecondObject(
    val transactionId: String
) {
    @field:JsonAnySetter
    @field:JsonAnyGetter
    val data: Map<String, Any?> = LinkedHashMap()
}

@JsonAutoDetect(
    fieldVisibility = JsonAutoDetect.Visibility.ANY,
    getterVisibility = JsonAutoDetect.Visibility.NONE
)
class ThirdObject(
    val transactionId: String,
    @field:JsonAnySetter
    @field:JsonAnyGetter
    val data: Map<String, Any?> = LinkedHashMap()
)

Input JSON for tests:

{
  "b": 2,
  "a": 1,
  "transactionId": "test",
  "c": [
    {
      "id": "3",
      "value": "c"
    },
    {
      "id": "1",
      "value": "a"
    },
    {
      "id": "2",
      "value": "b"
    }
  ]
}

Test Case:

Here’s a test case to reproduce the issue. The following Kotlin test serializes and deserializes three different classes (FirstObject, SecondObject, and ThirdObject), comparing the serialized JSON with the original input JSON.
JacksonSortingTest.kt branch 2.18.0.

For FirstObject, however, the test passed successfully. All fields were present in the serialized JSON, and they appeared in the expected order, as in the original input JSON.

Failing Test Results:

Expected:

{
  "b": 2,
  "a": 1,
  "transactionId": "test",
  "c": [
    {
      "id": "3",
      "value": "c"
    },
    {
      "id": "1",
      "value": "a"
    },
    {
      "id": "2",
      "value": "b"
    }
  ]
}

For SecondObject was:

{
  "transactionId": "test",
  "c": [
    {
      "id": "3",
      "value": "c"
    },
    {
      "id": "1",
      "value": "a"
    },
    {
      "id": "2",
      "value": "b"
    }
  ]
}

For ThirdObject was:

{
  "transactionId": "test"
}

Repository for Reproduction:
You can find a repository with the full reproduction of the issue at jackson-databind-sorting-issue - branch 2.18.0.

Expected behavior

After updating from Jackson 2.17.2 to 2.18.0, the serialization for SecondObject and ThirdObject should work as it did in the previous version (except for field ordering after deserialization and serialization). The expected JSON output after deserialization and serialization should contain all fields, with their values exactly as they are in the original input, regardless of field order.

Versions

Kotlin: 2.0.0
Jackson-module-kotlin: 2.18.0
Jackson-databind: 2.18.0

Additional context

This issue is a duplicate of 4752. I'm trying to reproduce it with Java, but it seems like all the described cases are specific to Kotlin

[k163377]

I was able to create a Java only reproduction based on the decompiled results.
Please re-open the databind issue or submit a new issue.

import com.fasterxml.jackson.annotation.*;
import com.fasterxml.jackson.annotation.JsonAutoDetect.Visibility;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;

import java.util.LinkedHashMap;
import java.util.Map;

public final class GitHub843Java {
    private final ObjectMapper objectMapper = new ObjectMapper();
    private final String jsonInput =
            "{\n" +
            "    \"b\": 2,\n" +
            "    \"a\": 1,\n" +
            "    \"transactionId\": \"test\",\n" +
            "    \"c\": [\n" +
            "        {\n" +
            "            \"id\": \"3\",\n" +
            "            \"value\": \"c\"\n" +
            "        },\n" +
            "        {\n" +
            "            \"id\": \"1\",\n" +
            "            \"value\": \"a\"\n" +
            "        },\n" +
            "        {\n" +
            "            \"id\": \"2\",\n" +
            "            \"value\": \"b\"\n" +
            "        }\n" +
            "    ]\n" +
            "}";

    private <T> void testSerializationDeserialization(Class<T> clazz) throws JsonProcessingException {
        Object deserializedObject = this.objectMapper.readValue(this.jsonInput, clazz);
        String serializedJson = this.objectMapper.writeValueAsString(deserializedObject);
        JsonNode expectedJson = this.objectMapper.readTree(this.jsonInput);
        JsonNode actualJson = this.objectMapper.readTree(serializedJson);
        Assertions.assertEquals(expectedJson, actualJson);
    }

    @Test
    public void test_serialization_and_deserialization_for_FirstObject() throws JsonProcessingException {
        this.testSerializationDeserialization(FirstObject.class);
    }

    @Test
    public void test_serialization_and_deserialization_for_SecondObject() throws JsonProcessingException {
        this.testSerializationDeserialization(SecondObject.class);
    }

    @Test
    public void test_serialization_and_deserialization_for_ThirdObject() throws JsonProcessingException {
        this.testSerializationDeserialization(ThirdObject.class);
    }

    @JsonAutoDetect(
            getterVisibility = Visibility.NONE,
            fieldVisibility = Visibility.ANY
    )
    public static final class FirstObject {
        @JsonAnySetter
        @JsonAnyGetter
        private final Map<String, Object> data = new LinkedHashMap<>();

        public Map<String, Object>  getData() {
            return this.data;
        }

        public String getTransactionId() {
            Object var1 = this.data.get("transactionId");
            return (String) var1;
        }
    }

    @JsonAutoDetect(
            getterVisibility = Visibility.NONE,
            fieldVisibility = Visibility.ANY
    )
    public static final class SecondObject {
        private final String transactionId;
        @JsonAnySetter
        @JsonAnyGetter
        private final Map<String, Object>  data;

        @JsonCreator
        public SecondObject(@JsonProperty("transactionId") String transactionId) {
            this.transactionId = transactionId;
            this.data = new LinkedHashMap<>();
        }

        public String getTransactionId() {
            return this.transactionId;
        }

        public Map<String, Object> getData() {
            return this.data;
        }
    }

    @JsonAutoDetect(
            getterVisibility = Visibility.NONE,
            fieldVisibility = Visibility.ANY
    )
    public static final class ThirdObject {
        private final String transactionId;
        @JsonAnySetter
        @JsonAnyGetter
        private final Map<String, Object> data;

        @JsonCreator
        public ThirdObject(
                @JsonProperty("transactionId") String transactionId,
                @JsonProperty("data") Map<String, Object> data
        ) {
            this.transactionId = transactionId;
            this.data = data;
        }

        public String getTransactionId() {
            return this.transactionId;
        }

        public Map<String, Object> getData() {
            return this.data;
        }
    }
}

[arvgord]

@k163377, thank you! I was able to fully reproduce the incorrect behavior in Java using your code (test file).

[arvgord]

I reopened the issue in jackson-databind.

[thejeff77]

So.. what version is the fix in? Can anyone post here?

[cowtowncoder]

@thejeff77 as per FasterXML/jackson-databind#4752 -- which was linked above and you may have noticed -- suggests relevant fix in 2.18.1.
But I would try 2.18.2 since that is the latest patch and has other possibly relevant fixes.