auto-value 1.10.2 breaks compatibility with Kotlin 1.6

[breun]

Upgrading auto-value from 1.10.1 to 1.10.2 in a Kotlin 1.6 project results in:

[ERROR] /path/to/.m2/repository/com/google/auto/value/auto-value/1.10.2/auto-value-1.10.2.jar!/META-INF/kotlin-stdlib-common.kotlin_module: (-1, -1) Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.

The auto-value 1.10.2 release notes don't mention any Kotlin compatibility changes, but compared to version 1.10.1 compatibility with Kotlin 1.6 and 1.7 was broken.

• auto-value 1.10.1 uses kotlinx-metadata-jvm 0.5.0, which uses kotlin-stdlib 1.7.0
• auto-value 1.10.2 uses kotlinx-metadata-jvm 0.6.2, which uses kotlin-stdlib 1.8.0
• auto-value HEAD-SNAPSHOT uses kotlinx-metadata-jvm 0.7.0, which uses kotlin-stdlib 1.9.0

Was compatibility with older Kotlin versions broken intentionally or accidentally?

[breun]

This backwards incompatible change indirectly caused Google Cloud BigQuery 2.31.0+ (google-cloud-bom 0.201.0+, libraries-bom 26.20.0+) and Spring Cloud GCP 3.6.1+ to break compatibility with Kotlin 1.6 and 1.7: GoogleCloudPlatform/spring-cloud-gcp#2122 (includes a link to a sample reproduction project)

[chaoren]

The kotlinx-metadata-jvm update happened automatically via @dependabot so it definitely wasn't intentional. @eamonnmcmanus how much do we care about old Kotlin versions? Would it be worthwhile to rollback the kotlinx-metadata-jvm updates?

[chaoren]

kotlinx-metadata-jvm 0.6.2 is required for Kotlin 1.9 support though. I don't think we want to drop support for the latest Kotlin version to support Kotlin 1.6.

[eamonnmcmanus]

I think there might be some subtlety here. If we use an old version of kotlinx-metadata-jvm, then will it be able to introspect the metadata from classes compiled with a more recent compiler? I gather version N can read the metadata from version N+1 but not necessarily N+2.

(I see @chaoren brought up essentially the same question.)

I'm not sure what the right solution is. Perhaps make the kotlinx-metadata-jvm dependency <scope>provided or <optional>true, so that it can be resolved to whatever the rest of the application is using? In that case we'd have to rewrite all the code using these imports so it goes through reflection, and behaves sensibly when the metadata classes aren't available.

[eamonnmcmanus]

Actually, if we do rewrite the metadata client code to use reflection, there's probably no reason to declare any sort of dependency on kotlinx-metadata-jvm. It will probably already be there if you're interacting with Kotlin data classes (which is what this is for), and if its API is backward-compatible we will find the right version to use and be able to use it.

I'll look into this possibility.

[eamonnmcmanus]

I did do the rewrite using reflection, but then realized that it isn't necessary. Many thanks to @breun for putting together the repro project. With that, I looked more closely at the error messages. As already noted in the original issue text, they look like this:

[ERROR] /path/to/.m2/repository/com/google/auto/value/auto-value/1.10.2/auto-value-1.10.2.jar!
/META-INF/kotlin-stdlib-common.kotlin_module: (-1, -1)
Module was compiled with an incompatible version of Kotlin.
The binary version of its metadata is 1.8.0, expected version is 1.6.0.

The issue is not with the copy of kotlinx-metadata-jvm (etc) in auto-value-1.10.2.jar. That is shaded, which means that the concerns I brought up in my earlier comment aren't real. The shading means that we can absolutely use the latest version of kotlinx-metadata-jvm without interfering with an earlier version that client code might be using. We just have to remove all the META-INF/*.kotlin_module entries from the jar. With that change, the repro project builds fine.

I'll look into getting a release out with that change in it shortly.

[eamonnmcmanus]

The newly-released 1.10.3 should fix this problem. Please let me know if not.

[breun]

Thanks for the quick fix and release, 1.10.3 indeed seems to fix this issue. 👍