Starvation situations with fixed thread pools

[nrinaudo]

Parallel collections that use a fixed thread pool for task support and contain "too many" elements will deadlock, where "too many" is a value I haven't been able to qualify.

The simplest possible reproduction is the empty list with a thread pool of 1:

import scala.concurrent.ExecutionContext
import java.util.concurrent.Executors
import scala.collection.parallel.CollectionConverters._
import scala.collection.parallel.ExecutionContextTaskSupport

val col = List.empty[Int].par

col.tasksupport = new ExecutionContextTaskSupport(
  ExecutionContext.fromExecutorService(Executors.newFixedThreadPool(1))
)

// This will deadlock
col.map(_ + 1)

I have observed the same starvation issue with larger thread pools and lists, but not reliably enough to provide a reproduction case.

[SethTisue]

@axel22 should https://docs.scala-lang.org/overviews/parallel-collections/configuration.html say something about this?

[axel22]

Maybe we should add a recommendation to use only ForkJoinPools to that documentation.

[nrinaudo]

Isn’t that hiding the symptoms though? Sure, it’s best to use a ForkJoinPool, but we still have perfectly legal code that deadlocks

[javax-swing]

This seems to stem from the implementation of

scala.collection.parallel.FutureTasks vs the implementation of scala.collection.parallel.ParIterableLike.ResultMapping

The issue is that

1. The FutureTasks logic first determines the parallelism based on the number of cores on the machine (in my case this was 6).

2. It then creates a computation tree based on the max depth calculated by this:
   private val maxdepth = (math.log(parallelismLevel) / math.log(2) + 1).toInt

3. This will create up to 2^maxDepth Futures (based on the number of elements in the collection)
   Each of this futures calls scala.collection.parallel.Task#tryLeaf

4. The tryLeaf method eventually calls through to scala.collection.parallel.Task#leaf. The implementation of leaf for ResultMapping scala.collection.parallel.ParIterableLike.ResultMapping#leaf makes a blocking call to tasksupport.executeAndWaitResult(inner)

5. In the case of a single thread, this means that the thread that is waiting for the task result is blocked because there is no thread available to process the scala.collection.parallel.ParIterableLike.StrictSplitterCheckTask

In theory, if the number of threads you define for your executor is:
(2 ^ (log(parallelismLevel) / log(2) + 1).toInt) + 1 then it shouldn't get deadlocked.... i think

Stack trace from an experiment.

java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
scala.concurrent.impl.Promise$DefaultPromise.tryAwait(Promise.scala:242)
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:258)
scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:263)
scala.concurrent.Await$.$anonfun$result$1(package.scala:219)
scala.concurrent.Await$$$Lambda$1274/950805155.apply(Unknown Source)
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:57)
scala.concurrent.Await$.result(package.scala:146)
scala.collection.parallel.FutureTasks.$anonfun$execute$3(Tasks.scala:513)
scala.collection.parallel.FutureTasks$$Lambda$1256/656809460.apply(Unknown Source)
scala.collection.parallel.FutureTasks.executeAndWaitResult(Tasks.scala:519)
scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult(Tasks.scala:555)
scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult$(Tasks.scala:555)
scala.collection.parallel.ExecutionContextTaskSupport.executeAndWaitResult(TaskSupport.scala:84)
scala.collection.parallel.ParIterableLike$ResultMapping.leaf(ParIterableLike.scala:960)
scala.collection.parallel.Task.$anonfun$tryLeaf$1(Tasks.scala:53)
scala.collection.parallel.Task$$Lambda$1257/1697135480.apply$mcV$sp(Unknown Source)
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:67)
scala.collection.parallel.Task.tryLeaf(Tasks.scala:56)
scala.collection.parallel.Task.tryLeaf$(Tasks.scala:50)
scala.collection.parallel.ParIterableLike$ResultMapping.tryLeaf(ParIterableLike.scala:955)
scala.collection.parallel.FutureTasks.$anonfun$exec$5(Tasks.scala:499)
scala.collection.parallel.FutureTasks$$Lambda$1253/1941666034.apply(Unknown Source)
scala.concurrent.Future$.$anonfun$apply$1(Future.scala:658)
scala.collection.parallel.FutureTasks$$Lambda$1254/1629179184.apply(Unknown Source)
scala.util.Success.$anonfun$map$1(Try.scala:255)
scala.util.Success.map(Try.scala:213)
scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
scala.concurrent.Future$$Lambda$1205/1115425820.apply(Unknown Source)
scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
scala.concurrent.impl.Promise$$Lambda$1206/183285557.apply(Unknown Source)
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)

[SethTisue]

Do we actually know whether @nrinaudo's original report is 2.12+-only?

I don't know for sure, but I suspect this ticket has become a grab bag of "parallel collections didn't work for me, in my code" reports that may or may not have anything to do with each other.

In particular, I suspect overlap with scala/bug#8119

[SethTisue]

I have hidden many comments which turned out to be unrelated to the original bug report.

Before commenting on this ticket, please be very sure that what you are encountering is exactly the specific issue that Nicolas has identified, rather than just any starvation or deadlock issue.