Support features of JDK 11+

[retronym]

In general, Scala now works just fine on JDK 11

For the current support status of each JDK version, see https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html

JDK9 JEP list

• JEP 223: New Version-String Scheme
  • Update our version parsing in isJavaAtLeast (and elsewhere?) SI-9750 scala.util.Properties.isJavaAtLeast works with JDK9 scala#5276
• JEP 220: Modular Run-Time Images (includes SI-9103)
  • Adapt our classpath to use jrt:/ virtual filesystem. WIP retronym:review/5112
• Update Zinc to understand that jrt:// classpath entries should have a timestamp based on the JDK (maybe $JAVA_HOME/jmods/*.mod? Find out exactly what file backs jrt://)
• Adapt REPL :javap command (currently gets IllegalArgumentException in JavapClass$JavapTool$.apply(JavapClass.scala:337)
• JEP-261 Module System (Support JEP-261 Module System #529)
  • javac allows programmers to declare module requirements, exports inline in the sourcetree ("module-info.java" files). What should we do?
  • app class loader is no longer a URLClassloader. Lots of places enumerate the classpath for JARs to initialize settings for a compiler. Experiment, document.
  • Update classfile reader to parse module-info.class (perhaps using ASM)
  • create module readability / accessibilty graph (following transitive "public exports")
• JEP-238 Multi-Release JAR Files
  • See if this feature is used in practice
  • Update classpath implementation to be able to consume such JARs. (WIP)
• JEP 280: Indify String Concatenation
  • Under target:jvm-1.9, consider emitting Javac style string concat with indy.
    • Use StringConcatFactory for string concatenation on JDK 9+ scala#9556
  • Update to ASM 5.1 (?). As suggested by @varming Varming/5.1 scala-asm-legacy#11
• JEP-193 VarHandles introduce a number of new methods annotated with @PolymorphicSignature. Expand our compiler support to these methods.
  • Maybe: foster creation of a library of Java 8/9 shims that use invokedynamic to use some Java 9 features (e.g VarHandle) when available but fall back to Java 8 alternatives (e.g. Unsafe._), as prototyped in retronym/scala@a95f03e
• Infrastructure:
  • Make JDK9 EA builds available in CI: Make it possible to use early access builds of OpenJDK scala-jenkins-infra#174
• SBT
  • Review Zinc's use of sun.boot.classpath: https://github.com/sbt/zinc/blob/1.x/internal/zinc-compile-core/src/main/scala/sbt/internal/inc/CompilerArguments.scala#L83
• Make mixin fields non-final to avoid verify errors. Use explicit fences to preserve memory effects: Trait encoding vs the persnickety JVM 9 verifier: final fields only updatable from within <init> #408

[retronym]

To whet the collective appetite, here's my branch that adapts to JEP 220 up and running:

⚡ qscala
Welcome to Scala 2.12.0-20160425-151134-5e07d57 (Java HotSpot(TM) 64-Bit Server VM, Java 9-ea).
Type in expressions for evaluation. Or try :help.

scala> java.util.List.of(1, 2, 3)
res0: java.util.List[Int] = [1, 2, 3]

[varming]

With less than nine months to the release of JDK9 not being able to run Scala programs on the early access releases is a serious concern, so it is great to see that work has started.

[SethTisue]

@retronym did 2.12.1 check some more of these boxes?

[retronym]

I have submitted a PR to SBT to start the ball rolling for Java 9 support. Because SBT internally uses an old version of scalac to compile build definitions, I decided to create a tool to export a jrt:/modules/*/** to rt.jar to bridge JDK9 to the older compiler.

This approach also enables your project to be compiled with, say, Scala 2.11.8, which might help some more folks test out JDK9, without first needing to upgrade to the 2.12 series.

Warning: we are yet to implement the additional access checks required by JEP-261. Until then, you can get LinkageError-s at runtime for code that passes the Scala type checker. So while I encourage testing and experimentation, use this at your own risk in production.

[retronym]

Discussion on the Java Language Spec changes related to modules, thread started by Eclipse JDT seeking to implement the spec. http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-May/012433.html

[ijuma]

Another Java 9 issue: scala/bug#10418

[SethTisue]

I've created https://github.com/scala/bug/labels/jdk9

[ijuma]

JEP 247: Compile for Older Platform Versions is also relevant. It would be nice if scalac could understand $JDK_ROOT/lib/ct.sym so that a newer JDK could be safely used to build for an older version.

[asarkar]

Is there a target release for addressing all the issues listed here, and reported by the users, or is it going to be on best-efforts basis?

[retronym]

@asarkar My primary goal at the moment is removing impediments to building and running Scala 2.12 on JDK9. We have been able to build and test the Play Framework, and the Akka team is working through similar testing at the moment.

I then want to implement support in the Scala typechecker for JPMS. I expect this will be done in Q4 this year. I'm hopeful that we could ship this in either a minor release of Scala 2.12, or as an experimental compiler plugin for Scala 2.12. It will introduce a dependency on the Java 9 standard library, which will complicate our build somewhat, as in general the compiler should only require Java 8.

I don't have a timeframe for the full list, but I'm keen to hear about what parts are most important for our users to help prioritize.

[smarter]

Update Zinc to understand that jrt:// classpath entries should have a timestamp based on the JDK (maybe $JAVA_HOME/jmods/*.mod? Find out exactly what file backs jrt://)

Surely that's not right for modules on the modulepath that are not part of the JDK?

[retronym]

Once we have a --modulepath compiler option, we'll be able to access the contents are regular JARs (well, apart from the .jmod format). So we might not actually access them via the jrt:// filesystem at all.

[lukaseder]

I managed to compile stuff on the JDK 9 using Scala 2.12.3, but not using 2.11.11 where I'm getting this error:

error: scala.reflect.internal.MissingRequirementError: object java.lang.Object in compiler mirror not found.

Is JDK 9 usage not supported with 2.11, or might there be a bug worth reporting?

[lrytz]

This is probably #304, fixed for 2.12 in scala/scala#6097. We're backporting some fixes for a 2.11.12 release, should be out soon.

[lukaseder]

Cook, thanks very much for the pointer, @lrytz !

[xuwei-k]

the main new thing is http://openjdk.java.net/jeps/181.

test/files/run/t7455/Test.scala fail on JDK11. (success on JDK10)

#559 (comment)

[SethTisue]

@retronym I'm retargeting this for 2.13.1, please open new tickets for any pieces that must make 2.13.0-RC1 or 2.13.0

[cekvenich]

It has been almost a year. Is there any ETA on JDK 11 support? Or we wait for Scala 3?

[smarter]

@cekvenich Scala 2 works on JDK 11 already, this bug report is about specific new features of the JDK that could be supported, are you interested in one of these features in particular?

[cekvenich]

@smarter No. I was just planning on installing Scala, and home pages says use Java 8, but I already have 11 and did not want issues. So should work w/11. OK, thanks.

[SethTisue]

perhaps it's time for us to tone down or remove the JDK 11 cautions at https://www.scala-lang.org/download/ and https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html#jdk-11-compatibility-notes . the current language in both locations was put in place quite some time ago

[SethTisue]

I've added "In general, Scala now works just fine on JDK 11" right at the top of this page.

I also changed the ticket title and tags to reflect that we don't care about 9 and 10 anymore, only 11 and higher.

[schrepfler]

Does the "just works" cover running on top of JDK 11 or also developing on top of it? I remember having some issues around module support with the satellite scala packages (java8-compat, scala-parser-combinators etc.) when trying to build stuff on it (I was on JDK 13 when I was trying this).

[nigredo-tori]

@schrepfler, I've been developing/running on top of JDK 11 for a while now (both with 2.11 and 2.12, and with some Java libraries in the mix). Most of the issues I encounter are caused by external libraries not built for Java 11 (e.g. ByteBuffer API has changed somewhere between 8 and 11), missing modules (solved by adding them to classpath as managed/unmanaged dependencies), and class loader changes. JPMS is not an issue unless you want to really dig into it and build your own modules.

Basically, nothing prevents you from working with JDK 11. You'll encounter some annoying issues - but those are not unique to Scala. Some JPMS-related features are not supported - but everything that was possible with JDK 8 should be possible with JDK 11 after some fiddling.

[SethTisue]

@schrepfler it's not uncommon for a random Scala OSS repo to fail to build on JDK 13 without tweaks, usually trivial ones like bumping the SBT version or disambiguating a now-ambiguous import, that kind of thing. so if you're working in other people's repos, you'll probably want to have JDK 8 and/or 11 on your system as a fallback plan, in case the needed tweaks aren't obvious

some repos may also fail to build on 11, but I think that has become much less common

I agree with @nigredo-tori 's remarks as well.

[raboof]

external libraries not built for Java 11 (e.g. ByteBuffer API has changed somewhere between 8 and 11)

(kindof offtopic, but since it was mentioned: if a library has a problem with ByteBuffer.clear when running under JDK8, this is likely because it was built with JDK11 targeting JDK8, but using the JDK11 bootclasspath. Using the --release/-release flags for javac/scalac while compiling the library that calls ByteBuffer.clear should fix this, but IIRC there was some problem with this on scalac versions before 2.12.8/2.13.1)