Update Scala to 2.13 #1522

poslegm · 2019-10-09T21:44:18Z

Fixes #1434

Aaaand now pipeline is green 🚀

Please take a look at ScalafmtVersion introduced in this PR. Now it's difficult to compare scalafmt versions in ScalafmtDynamicDownloader only with manual string parsing (e.g. .startsWith("0.1")).
Hard-coded scalafmt version with Scala 2.13 in ScalafmtDynamicDownloader is 2.2.0. Is it ok? UPD: 2.1.2
There are a lot of garbage commits in my branch. I don't like to force push into already published branches, so squash it on merge, please 😄

scalafmt-core/shared/src/main/scala/org/scalafmt/internal/FormatOps.scala

olafurpg

Thank you for looking into this! Just a few minor comments, otherwise looks great!

olafurpg · 2019-10-10T05:36:02Z

build.sbt

@@ -3,6 +3,7 @@ import sbtcrossproject.CrossPlugin.autoImport.crossProject

 def scala211 = "2.11.12"
 def scala212 = "2.12.8"
+def scala213 = "2.13.0"


Suggested change

def scala213 = "2.13.0"

def scala213 = "2.13.1"

Unpleasant discovery: scalafmt should have similar Scala version with coursier-interface for assembly working 😒

Can you elaborate? I thought scala-library was shaded with coursier-interface

The problem in reflect.properties file (#1514 (comment)). coursier-interface jar contains file with content:

#Fri Jun 07 10:55:45 UTC 2019 shell.banner=%n ________ ___ / / ___%n / __/ __// _ | / / / _ |%n __\\ \\/ /__/ __ |/ /__/ __ |%n /____/\\___/_/ |_/____/_/ | |%n |/ %s copyright.string=Copyright 2002-2019, LAMP/EPFL and Lightbend, Inc. version.number=2.13.0 osgi.version.number=2.13.0.v20190604-151517-VFINAL-43e040f maven.version.number=2.13.0

But I hope that we can add sbt-assembly rule to discard scala-reflect from coursier-interface.

It should be fine to discard the reflect.properties file from coursier-interface, unless the code from that scala-library somehow relies on APIs like scala.util.Properties.versionNumberString.

build.sbt

scalafmt-cli/src/main/scala/org/scalafmt/cli/ScalafmtCoreRunner.scala

scalafmt-core/shared/src/main/scala/org/scalafmt/internal/Router.scala

tanishiking

Awesome! Great work @poslegm 🎉
I left a minor comment, but others look good.

What do you think about scalatags, which introduced scala 2.13 support and dropped 2.11 support in 0.7.0?

scalafmt/build.sbt

Line 154 in fea0efc

"com.lihaoyi" %% "scalatags" % "0.6.8",

This library is used only here for writing rich test report like this.
Though this would be helpful for scalafmt's optimization, maybe we can drop this feature for now. If we need this report later, we can revive this report somehow (by writing raw HTML or using another template engine).

scalafmt-cli/src/main/scala/org/scalafmt/cli/ScalafmtCoreRunner.scala

olafurpg · 2019-10-10T19:04:46Z

The trick to handle older versions of scalatags is to add the following to build.sbt

lazy val scalatagsVersion = Def.setting {
  scalaBinaryVersion.value match {
    case "2.11" => "0.6.8"
    case _ => "LATEST.."
  }
}

libraryDependencies += "com.lihaoyi" %% "scalatags" % scalatagsVersion.value

poslegm · 2019-10-13T13:44:27Z

@olafurpg @tanishiking

Current status

CI for 2.13 fails with

0.98s$ sbt ci-test
[info] [launcher] getting org.scala-sbt sbt 1.3.2  (this may take some time)...
:: loading settings :: url = jar:file:/home/travis/.sbt/launchers/1.3.2/sbt-launch.jar!/org/apache/ivy/core/settings/ivysettings.xml
:: retrieving :: org.scala-sbt#boot-app
	confs: [default]
:: loading settings :: url = jar:file:/home/travis/.sbt/launchers/1.3.2/sbt-launch.jar!/org/apache/ivy/core/settings/ivysettings.xml
	1 artifacts copied, 0 already retrieved
[error] [launcher] error during sbt launcher: sbt/sbt#4955!
[error] [launcher] sbt launcher is unable to detect the Scala version for org.scala-sbt:sbt;
[error] [launcher] this likely indicates an incomplete artifact resolution and/or corrupt boot cache (~/.sbt/boot/).
[error] [launcher] the following is the full classpath that was retrieved for org.scala-sbt:sbt:
[error] [launcher]  * /home/travis/.sbt/boot/other/org.scala-sbt/sbt/1.3.2/sbt-1.3.2.jar

Benchmarks module doesn't compile because of Corpus from scalameta testkit. It available for 2.12 only.

[error] /Users/poslegm/Documents/scalafmt/scalafmt-benchmarks/src/main/scala/org/scalafmt/benchmarks/MacroBenchmark.scala:5:8: object Corpus is not a member of package scala.meta.testkit
[error] import scala.meta.testkit.Corpus
[error]        ^
[error] /Users/poslegm/Documents/scalafmt/scalafmt-benchmarks/src/main/scala/org/scalafmt/benchmarks/MacroBenchmark.scala:6:8: object CorpusFile is not a member of package scala.meta.testkit
[error] import scala.meta.testkit.CorpusFile
[error]        ^
[error] /Users/poslegm/Documents/scalafmt/scalafmt-benchmarks/src/main/scala/org/scalafmt/benchmarks/MacroBenchmark.scala:61:15: not found: value Corpus
[error]       val x = Corpus
[error]               ^
[error] /Users/poslegm/Documents/scalafmt/scalafmt-benchmarks/src/main/scala/org/scalafmt/benchmarks/MacroBenchmark.scala:63:11: not found: value Corpus
[error]           Corpus.fastparse.copy(
[error]           ^
[error] /Users/poslegm/Documents/scalafmt/scalafmt-benchmarks/src/main/scala/org/scalafmt/benchmarks/MacroBenchmark.scala:65:13: not found: value url
[error]             url = Corpus.fastparse.url.replace("olafurpg", "scalameta")
[error]             ^
[error] 5 errors found

DynamicSuite and CliTest fails with

[info] tests.DynamicSuite *** ABORTED ***
[info]   java.lang.NoClassDefFoundError: scala/Serializable
[info]   at java.base/java.lang.ClassLoader.defineClass1(Native Method)
[info]   at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
[info]   at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
[info]   at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:515)
[info]   at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:423)
[info]   at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:417)
[info]   at java.base/java.security.AccessController.doPrivileged(AccessController.java:689)
[info]   at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:416)
[info]   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
[info]   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
[info]   ...
[info]   Cause: java.lang.ClassNotFoundException: scala.Serializable
[info]   at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)
[info]   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
[info]   at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
[info]   at java.base/java.lang.ClassLoader.defineClass1(Native Method)
[info]   at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
[info]   at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
[info]   at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:515)
[info]   at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:423)
[info]   at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:417)
[info]   at java.base/java.security.AccessController.doPrivileged(AccessController.java:689)
[info]   ...
[error] Test suite tests.DynamicSuite failed with java.lang.NoClassDefFoundError: scala/Serializable.
[error] This may be due to the ClassLoaderLayeringStrategy (ScalaLibrary) used by your task.
[error] To improve performance and reduce memory, sbt attempts to cache the class loaders used to load the project dependencies.
[error] The project class files are loaded in a separate class loader that is created for each test run.
[error] The test class loader accesses the project dependency classes using the cached project dependency classloader.
[error] With this approach, class loading may fail under the following conditions:
[error]
[error]  * Dependencies use reflection to access classes in your project's classpath.
[error]    Java serialization/deserialization may cause this.
[error]  * An open package is accessed across layers. If the project's classes access or extend
[error]    jvm package private classes defined in a project dependency, it may cause an IllegalAccessError
[error]    because the jvm enforces package private at the classloader level.
[error]
[error] These issues, along with others that were not enumerated above, may be resolved by changing the class loader layering strategy.
[error] The Flat and ScalaLibrary strategies bundle the full project classpath in the same class loader.
[error] To use one of these strategies, set the  ClassLoaderLayeringStrategy key
[error] in your configuration, for example:
[error]
[error] set dynamic / Test / classLoaderLayeringStrategy := ClassLoaderLayeringStrategy.ScalaLibrary
[error] set dynamic / Test / classLoaderLayeringStrategy := ClassLoaderLayeringStrategy.Flat
[error]
[error] See ClassLoaderLayeringStrategy.scala for the full list of options.

olafurpg · 2019-10-13T21:28:35Z

@poslegm it's fine to move the benchmark project to the scala-2.12 source directory.

Can you try downgrading sbt to 1.2.8? Those errors look 1.3.2 related.

poslegm · 2019-10-14T10:22:03Z

Ok, issues 1 and 2 are solved. Thank you @olafurpg! But java.lang.NoClassDefFoundError: scala/Serializable is still here (look at CI).

olafurpg · 2019-10-15T09:47:09Z

@poslegm the CI is green, is the Serializable error still a problem?

poslegm · 2019-10-15T10:39:33Z

@olafurpg No, I fixed it. The problem was in Scala version for artifact in ScalafmtDynamicDownloader

tanishiking

@poslegm Thank you so much for this huge effort! 😄
I left several comments mainly on the version parser.

.travis.yml

scalafmt-dynamic/src/main/scala/org/scalafmt/dynamic/ScalafmtVersion.scala

scalafmt-dynamic/src/main/scala/org/scalafmt/dynamic/ScalafmtDynamicDownloader.scala

tanishiking · 2019-10-15T15:57:03Z

scalafmt-dynamic/src/main/scala/org/scalafmt/dynamic/ScalafmtVersion.scala

+
+import scala.util.control.{NonFatal, NoStackTrace}
+
+case class ScalafmtVersion(major: Int, minor: Int, patch: Int, rc: Int) {


[note] Simple implementation of semantic versioning, this is beautiful 👍
(If we have some tried and trusted semver library, maybe we should replace this logic with the library for simplifying the logic, but I'm not sure)

tanishiking · 2019-10-15T16:15:34Z

scalafmt-dynamic/src/test/scala/tests/ScalafmtVersionSuite.scala

+    assert(
+      ScalafmtVersion
+        .parse("1.2.3-M14") == Left(InvalidVersionException("1.2.3-M14"))
+    )


Could you add some more invalid examples? like

2.-1.0 (we could check all the fields are positive or 0)

2.1.0.

2.1.0.1

2.1.0-rc1 (small letter)

Added tests. Also I rewrote parser into regex because it's difficult to handle all invalid cases manually.

scalafmt-dynamic/src/main/scala/org/scalafmt/dynamic/ScalafmtDynamic.scala

olafurpg

This is a significant contribution! Thank you @poslegm for your work. LGTM 👍

tanishiking · 2019-10-17T17:02:38Z

Awesome! Thank you so much for this big contribution @poslegm ! LGTM
I'll release 2.2.0-RC1 tomorrow, and if there's looks no problem, gonna release 2.2.0 🎉

First steps to 2.13

07c654b

poslegm commented Oct 9, 2019

View reviewed changes

scalafmt-core/shared/src/main/scala/org/scalafmt/internal/FormatOps.scala Outdated Show resolved Hide resolved

olafurpg reviewed Oct 10, 2019

View reviewed changes

poslegm added 3 commits October 10, 2019 19:10

Macros fixed for 2.12 and 2.11

09ef028

Formatting

bcc0c3b

Travis job for 2.13

c6b6f55

tanishiking reviewed Oct 10, 2019

View reviewed changes

scalafmt-cli/src/main/scala/org/scalafmt/cli/ScalafmtCoreRunner.scala Outdated Show resolved Hide resolved

poslegm added 7 commits October 11, 2019 23:20

Parallel collections, scalatags and mdoc

a980e56

Scalameta downgrade

2122fdd

Token => T

1005027

Tests compilation fix

0fd77b0

.sbt cache disabled

27f61ae

Try to fix CI and small refactoring

7b2dc99

Fixed par collections and ANSI literal

736c436

Benchmarks moved to 2.12 directory, sbt downgraded

62fb8db

poslegm added 3 commits October 15, 2019 00:35

ScalafmtVersion introduced

d087314

Fixed classpath errors for scala 2.12 artifacts

e3c6c97

Fixed RC handling in ScalafmtVersion

82526fd

poslegm marked this pull request as ready for review October 14, 2019 22:13

poslegm changed the title ~~First steps to 2.13~~ Update Scala to 2.13 Oct 14, 2019

poslegm requested review from olafurpg and tanishiking October 14, 2019 22:13

tanishiking reviewed Oct 15, 2019

View reviewed changes

Reverted travis CI cache

48ddad0

poslegm added 2 commits October 16, 2019 19:02

Changed versions in ScalafmtDynamicDownloader

82e36e0

ScalafmtVersion rewrote into regex

6dc2a22

poslegm requested a review from tanishiking October 16, 2019 16:32

olafurpg approved these changes Oct 17, 2019

View reviewed changes

olafurpg merged commit e8a1aec into scalameta:master Oct 17, 2019

poslegm deleted the scala213 branch October 17, 2019 13:33

tanishiking mentioned this pull request Nov 13, 2019

Make it easier to test scalafmt local snapshot build #1552

Merged

tgodzik mentioned this pull request Mar 26, 2021

Update scalatags to 0.9.4 #2387

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Scala to 2.13 #1522

Update Scala to 2.13 #1522

poslegm commented Oct 9, 2019 •

edited

Loading

olafurpg left a comment

olafurpg Oct 10, 2019

poslegm Oct 10, 2019

olafurpg Oct 10, 2019

poslegm Oct 10, 2019

olafurpg Oct 13, 2019

tanishiking left a comment

olafurpg commented Oct 10, 2019

poslegm commented Oct 13, 2019

olafurpg commented Oct 13, 2019

poslegm commented Oct 14, 2019

olafurpg commented Oct 15, 2019

poslegm commented Oct 15, 2019

tanishiking left a comment

tanishiking Oct 15, 2019

tanishiking Oct 15, 2019 •

edited

Loading

poslegm Oct 16, 2019

olafurpg left a comment

tanishiking commented Oct 17, 2019 •

edited

Loading


		import scala.util.control.{NonFatal, NoStackTrace}

		case class ScalafmtVersion(major: Int, minor: Int, patch: Int, rc: Int) {

Update Scala to 2.13 #1522

Update Scala to 2.13 #1522

Conversation

poslegm commented Oct 9, 2019 • edited Loading

olafurpg left a comment

Choose a reason for hiding this comment

olafurpg Oct 10, 2019

Choose a reason for hiding this comment

poslegm Oct 10, 2019

Choose a reason for hiding this comment

olafurpg Oct 10, 2019

Choose a reason for hiding this comment

poslegm Oct 10, 2019

Choose a reason for hiding this comment

olafurpg Oct 13, 2019

Choose a reason for hiding this comment

tanishiking left a comment

Choose a reason for hiding this comment

olafurpg commented Oct 10, 2019

poslegm commented Oct 13, 2019

Current status

olafurpg commented Oct 13, 2019

poslegm commented Oct 14, 2019

olafurpg commented Oct 15, 2019

poslegm commented Oct 15, 2019

tanishiking left a comment

Choose a reason for hiding this comment

tanishiking Oct 15, 2019

Choose a reason for hiding this comment

tanishiking Oct 15, 2019 • edited Loading

Choose a reason for hiding this comment

poslegm Oct 16, 2019

Choose a reason for hiding this comment

olafurpg left a comment

Choose a reason for hiding this comment

tanishiking commented Oct 17, 2019 • edited Loading

poslegm commented Oct 9, 2019 •

edited

Loading

tanishiking Oct 15, 2019 •

edited

Loading

tanishiking commented Oct 17, 2019 •

edited

Loading