Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to support 2.12/2.13 cross-building? #22

Closed
SethTisue opened this issue Mar 22, 2017 · 9 comments
Closed

how to support 2.12/2.13 cross-building? #22

SethTisue opened this issue Mar 22, 2017 · 9 comments

Comments

@SethTisue
Copy link
Member

SethTisue commented Mar 22, 2017

hey @szeiger if I need import scala.collection.parallel.CollectionConverters._ before I can use .par then how are library maintainers supposed to cross-build their code in 2.12/2.13?

idea: should we add a dummy empty scala.collection.parallel.CollectionConverters class to Scala 2.12.2? if it has no runtime residue, doing so won't break binary compatibility as far as I can see

@retronym
Copy link
Member

This might be a good usecase for -Ypredef, you could have Scala version dependent root imports.

@szeiger
Copy link
Contributor

szeiger commented Mar 22, 2017

For now I would suggest to put a dummy CollectionConverters object into a scala-2.12 source folder in affected projects. It's a one-line change and sbt supports it out of the box.

In the long run we'll need some compatibility library for 2.12 -> 2.13 collections migration where we can add this object.

@sjrd
Copy link
Member

sjrd commented Mar 22, 2017

@szeiger That solution is not really acceptable: it leaks that dummy object down to other projects depending on your library. And if there are two such libraries, you now inherit from twice this dummy object on the classpath. That's too ugly to be a recommended default solution.

@sjrd
Copy link
Member

sjrd commented Mar 22, 2017

Here is a non-leaking solution, fully source compatible across Scala versions (no need for scala-2.12 or scala-2.13 folders).

CompatParColls.scala (private to the library)

package test

private[test] object CompatParColls {
  val Converters = {
    import Compat._

    {
      import scala.collection.parallel._

      CollectionConverters
    }
  }

  object Compat {
    object CollectionConverters
  }
}

Test.scala (actual code in the library that wants to use parallel collections)

package test

import test.CompatParColls.Converters._

object Main {
  def main(args: Array[String]): Unit = {
    val argsInt = args.par.map(_.toInt)
    argsInt.foreach(println(_))
  }
}

Gist: https://gist.github.com/sjrd/ccc2ec61a2ccbae1eb4ee9dea004b7a5

@SethTisue
Copy link
Member Author

note that it's often possible to replace something like

Vector.fill(100)(...).par

with

collection.parallel.immutable.ParVector.fill(100)(...)

which is fully source-compatible.

@szeiger
Copy link
Contributor

szeiger commented May 7, 2018

We should provide 2.11 & 2.12 builds with a dummy CollectionConverters objects. We're already using the same approach for https://github.com/scala/scala-collection-compat/

35VLG84 added a commit to e257-fi/tackler that referenced this issue Aug 8, 2019
srowen added a commit to apache/spark that referenced this issue Oct 3, 2019
…sier; add scala-2.13 profile to enable pulling in par collections library separately, for the future

### What changes were proposed in this pull request?

Scala 2.13 removes the parallel collections classes to a separate library, so first, this establishes a `scala-2.13` profile to bring it back, for future use.

However the library enables use of `.par` implicit conversions via a new class that is not in 2.12, which makes cross-building hard. This implements a suggested workaround from scala/scala-parallel-collections#22 to avoid `.par` entirely.

### Why are the changes needed?

To compile for 2.13 and later to work with 2.13.

### Does this PR introduce any user-facing change?

Should not, no.

### How was this patch tested?

Existing tests.

Closes #25980 from srowen/SPARK-29296.

Authored-by: Sean Owen <[email protected]>
Signed-off-by: Sean Owen <[email protected]>
@SethTisue
Copy link
Member Author

Having now gone so long without providing empty 2.11 and 2.12 artifacts, my conclusion is that there isn't much demand and people are coping fine with what they have.

@christianrua

This comment was marked as off-topic.

@SethTisue

This comment was marked as off-topic.

eed3si9n added a commit to eed3si9n/sbt that referenced this issue Jan 11, 2021
Ref scala/scala-parallel-collections#22

Parallel collection got split off without source-compatible library, so apparently we need to roll our own compat hack, which causes import not used, so it needs to be paired with silencer.
@SethTisue SethTisue closed this as not planned Won't fix, can't repro, duplicate, stale Sep 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants
@szeiger @retronym @SethTisue @sjrd @christianrua and others