rewrite-java-21 (8.36.0) fails on Java 23

[Col-E]

Overview

#4534 was closed under the assumption that the reporter's problem was their project was picking up the Java 17 parser and not the patched Java 21 parser (with #4516))

Unfortunately, even with the provided patched rewrite-java-21 parsing code on Java 23 fails with an exception:

java.lang.NoSuchMethodError: 'com.sun.tools.javac.tree.DCTree$DCDocComment com.sun.tools.javac.tree.DocCommentTable.getCommentTree(com.sun.tools.javac.tree.JCTree)'
  org.openrewrite.java.isolated.ReloadableJava21ParserVisitor.convert(ReloadableJava21ParserVisitor.java:1667)
  org.openrewrite.java.isolated.ReloadableJava21ParserVisitor.visitCompilationUnit(ReloadableJava21ParserVisitor.java:538)
  org.openrewrite.java.isolated.ReloadableJava21ParserVisitor.visitCompilationUnit(ReloadableJava21ParserVisitor.java:74)
  com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:625)
  com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:92)

The problem

The line in question is here: ReloadableJava21ParserVisitor.java#L1667

• The rewrite-java-21 module is sensibly compiled against JDK 21
• This means the reference to DocCommentTable.getCommentTree is expecting the return type to be what was defined in the JDK 21 code
• Running rewrite-java-21 on JDK 23 crashes because the getCommentTree changed the return type in a non-binary compatible way
  • JDK 22 - DocCommentTable.java#L69 - return type is com.sun.tools.javac.tree.DCTree.DCDocComment
  • JDK 23 - DocCommentTable.java#L69 - return type is com.sun.source.doctree.DocCommentTree
  • Compiling against 22 or below makes method reference with a return type DCDocComment which cannot be coerced to be the looser DocCommentTree that would be compatible on newer JDK's

Solutions

• Create a rewrite-java-23 module
• Compile rewrite-java-21 against JDK 23 but with a target version of Java 21
  • This should emit a method reference to the method with a looser return type, and since the older Java code's return value implements the interface type of the new methods return type it should be compatible
  • Assuming the rewrite-java-21 is thoroughly covered, any other possible changes from targeting the newer API (with a compatible bytecode output though) that are incompatible should be caught by the tests
• Use reflection to support use from both <23 and >=23

Reflection solution:

// Lookup by name does not require strict contract with return type
Method getCommentTreeMethod = DocCommentTable.class.getMethod("getCommentTree", JCTree.class);

// DCTree.DCDocComment implements DocCommentTree so the cast is safe in both 22- and 23+
DocCommentTree value = (DocCommentTree) getCommentTreeMethod.invoke(table, (Object) null);

Workarounds

• Run your application on Java 22 or lower

[timtebeek]

Thanks a lot for pulling together all the relevant context here @Col-E , and outlining possible next steps. I'll pass this along to colleagues as I'm traveling, but if you want to push up the reflection fix that ought to be manageable to review on the go. I don't see us adding a Java 23 specific parser for now, as it's quite a bit to maintain for a non-LTS version.

[timtebeek]

I've pushed up this proposal based on your last suggestion above

• Use reflection to invoke getCommentTree to support Java 21 & 23 #4555

I hope I understood you correctly @Col-E ! :)