test sbt compatibility via GH matrix rather than dedicated scripted

[bjaglin]

Trying #193 (comment)

There are now 3 versions of sbt involved:

• The one used to build the project
• The one we build the plugin against
• The ones we run scripted tests with

[thesamet]

Thanks for following up on this! One suggestions is to disable fail fast so we can see all errors at once: https://docs.github.com/en/free-pro-team@latest/actions/reference/workflow-syntax-for-github-actions#jobsjob_idstrategyfail-fast

[bjaglin]

Looking at

2020-11-04T19:02:54.6353428Z Running settings / cross-platform
2020-11-04T19:02:54.9854426Z [error] Getting org.scala-sbt sbt 1.1.6  (this may take some time)...
2020-11-04T19:02:56.1854116Z [info] :: retrieving :: org.scala-sbt#boot-app
2020-11-04T19:02:56.1899221Z [info] 	confs: [default]
2020-11-04T19:02:56.2554546Z [info] 	76 artifacts copied, 0 already retrieved (27669kB/71ms)
2020-11-04T19:02:56.2652122Z [error] Getting Scala 2.12.6 (for sbt)...
2020-11-04T19:02:56.3139620Z [info] :: retrieving :: org.scala-sbt#boot-scala
2020-11-04T19:02:56.3171846Z [info] 	confs: [default]
2020-11-04T19:02:56.3370662Z [info] 	5 artifacts copied, 0 already retrieved (19632kB/23ms)
2020-11-04T19:02:58.7115368Z [info] [info] Updated file /tmp/sbt_82ae5404/cross-platform/project/build.properties: set sbt.version to 1.1.6
2020-11-04T19:03:03.0070507Z [info] [info] Loading settings from plugins.sbt ...
2020-11-04T19:03:03.7220949Z [info] [info] Loading project definition from /tmp/sbt_82ae5404/cross-platform/project
2020-11-04T19:03:03.9805931Z [info] [info] Updating ProjectRef(uri("file:/tmp/sbt_82ae5404/cross-platform/project/"), "cross-platform-build")...
2020-11-04T19:03:05.5982417Z [info] [warn] Choosing local for com.thesamet#sbt-protoc;1.0.0-SNAPSHOT
2020-11-04T19:03:24.4119782Z [info] [info] Done updating.
2020-11-04T19:03:24.4310332Z [info] [warn] Found version conflict(s) in library dependencies; some are suspected to be binary incompatible:
2020-11-04T19:03:24.4312083Z [info] [warn] 	* com.google.protobuf:protobuf-java:3.5.1 is selected over 2.5.0
2020-11-04T19:03:24.4313024Z [info] [warn] 	    +- com.thesamet.scalapb:compilerplugin_2.12:0.7.4     (depends on 3.5.1)
2020-11-04T19:03:24.4313902Z [info] [warn] 	    +- org.scala-js:closure-compiler-java-6:v20160517     (depends on 2.5.0)
2020-11-04T19:03:24.4331275Z [info] [warn] 	* com.thesamet.scalapb:protoc-bridge_2.12:0.9.0-RC6 is selected over 0.7.3
2020-11-04T19:03:24.4332318Z [info] [warn] 	    +- com.thesamet:sbt-protoc:1.0.0-SNAPSHOT (scalaVersion=2.12, sbtVersion=1.0) (depends on 0.7.3)
2020-11-04T19:03:24.4333211Z [info] [warn] 	    +- com.thesamet.scalapb:compilerplugin_2.12:0.7.4     (depends on 0.7.3)
2020-11-04T19:03:24.4334067Z [info] [warn] 	* org.portable-scala:sbt-platform-deps:1.0.0 is selected over 1.0.0-M2
2020-11-04T19:03:24.4335088Z [info] [warn] 	    +- org.portable-scala:sbt-scalajs-crossproject:0.6.0 (scalaVersion=2.12, sbtVersion=1.0) (depends on 1.0.0)
2020-11-04T19:03:24.4336158Z [info] [warn] 	    +- com.thesamet:sbt-protoc:1.0.0-SNAPSHOT (scalaVersion=2.12, sbtVersion=1.0) (depends on 1.0.0)
2020-11-04T19:03:24.4337129Z [info] [warn] 	    +- org.portable-scala:sbt-crossproject:0.6.0 (scalaVersion=2.12, sbtVersion=1.0) (depends on 1.0.0)
2020-11-04T19:03:24.4338256Z [info] [warn] 	    +- org.portable-scala:sbt-scala-native-crossproject:0.6.0 (scalaVersion=2.12, sbtVersion=1.0) (depends on 1.0.0)
2020-11-04T19:03:24.4339299Z [info] [warn] 	    +- org.scala-js:sbt-scalajs:0.6.23 (scalaVersion=2.12, sbtVersion=1.0) (depends on 1.0.0)
2020-11-04T19:03:24.4340231Z [info] [warn] 	    +- org.scala-native:sbt-scala-native:0.3.7 (scalaVersion=2.12, sbtVersion=1.0) (depends on 1.0.0-M2)
2020-11-04T19:03:24.4341049Z [info] [warn] Run 'evicted' to see detailed eviction warnings
2020-11-04T19:03:25.6898255Z [error] /tmp/sbt_82ae5404/cross-platform/build.sbt:5: error: Symbol 'type sbt.CompositeProject' is missing from the classpath.
2020-11-04T19:03:25.6900342Z [error] This symbol is required by 'type sbtcrossproject.SBTCompat.CompositeProject'.
2020-11-04T19:03:25.6902098Z [error] Make sure that type CompositeProject is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
2020-11-04T19:03:25.6903495Z [error] A full rebuild may help if 'SBTCompat.class' was compiled against an incompatible version of sbt.
2020-11-04T19:03:25.6904600Z [error] lazy val crossPlatform = crossProject(JSPlatform, JVMPlatform, NativePlatform)
2020-11-04T19:03:25.6905430Z [error]                                      ^
2020-11-04T19:03:25.9529888Z [info] [error] sbt.compiler.EvalException: Type error in expression
2020-11-04T19:03:25.9536996Z [info] [error] sbt.compiler.EvalException: Type error in expression
2020-11-04T19:03:25.9542302Z [info] [error] Use 'last' for the full log.
2020-11-04T19:03:26.3233776Z [info] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? 

[thesamet]

I am personally ok with supporting only sbt 1.2.x through 1.4.x. Seems a pretty reasonable range.

[bjaglin]

@thesamet agreed - some users might be stuck on 1.2.x because of coursier (even though it can be disabled in 1.3.x), but I can't think of any reason to be stuck on 1.0.x or 1.1.x, so I am limiting the matrix to 1.2.8+.

By the way, I think the only problem with pre-1.2.x was that https://github.com/portable-scala/sbt-crossproject used in settings/cross-platform is 1.2.1+, so sbt-protoc might be fine, it was just the scripted itself that could not load. It's possible to conditionally skipping that test if we really want to ensure pre-1.2.x compatibility, but again, there is not strong driver for that.

[bjaglin]

I see mergify is set up here, so I am not sure this comment really helps. Can mergify ignore that specific bump maybe?

[thesamet]

Nothing needs to be updated with mergify. The trick we're using is to create a single "ci-passed" job that runs at the very end only if all tests pass:

sbt-protoc/.github/workflows/ci.yml

Lines 45 to 49 in adbb448

	ci-passed:
	runs-on: ubuntu-latest
	needs: build
	steps:
	- run: ':'

and mergify awaits only on this.

[bjaglin]

Sorry if I wasn't clear: what I meant is that the scala-steward sbt bumps will be merged automatically, so we won't get the opportunity to keep the GH scripted-sbt array values up to date as sbt 1.4.x patch releases roll out, like this comment suggests.

[thesamet]

Thanks!