Threads appears to be unstable

[larskanis]

While testing Eventbox on Truffleruby I noticed all kinds of errors due to the heavy threads usage. I'm unsure whether Truffleruby currently intends to be threadsafe and whether it intends to follow the concurrency rules of JRuby. As a first example here is a small script that raises a Java error with a 50% probability. I'll try to break the problems into more such failing scripts, if this is desired.

1000.times.map do
  Thread.new do
    Thread.new do
    end.join
  end
end.each(&:join)

Fails with:

java.lang.UnsupportedOperationException: makeSharedShape() can only be called on non-shared shapes.
	at java.lang.Throwable.<init>(Throwable.java:265)
	at java.lang.Exception.<init>(Exception.java:66)
	at java.lang.RuntimeException.<init>(RuntimeException.java:62)
	at java.lang.UnsupportedOperationException.<init>(UnsupportedOperationException.java:52)
	at com.oracle.truffle.object.ShapeImpl.makeSharedShape(ShapeImpl.java:956)
	at org.truffleruby.language.objects.shared.SharedObjects.share(SharedObjects.java:130)
	at org.truffleruby.language.objects.shared.SharedObjects.shareObjects(SharedObjects.java:90)
	at org.truffleruby.language.objects.shared.SharedObjects.shareContextRoots(SharedObjects.java:66)
	at org.truffleruby.language.objects.shared.SharedObjects.startSharing(SharedObjects.java:41)
	at org.truffleruby.core.thread.ThreadManager.registerThread(ThreadManager.java:510)
	at org.truffleruby.core.thread.ThreadManager.start(ThreadManager.java:286)
	at org.truffleruby.core.thread.ThreadManager.threadMain(ThreadManager.java:240)
	at org.truffleruby.core.thread.ThreadManager.lambda$initialize$3(ThreadManager.java:230)
	at org.truffleruby.core.thread.ThreadManager$$Lambda$1146/1602795366.run(Unknown Source)
	at java.lang.Thread.run(Thread.java:748)
	at com.oracle.truffle.polyglot.PolyglotThread.run(PolyglotThread.java:70)
	at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:476)
	at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:199)

Tested on

$ ruby -v
truffleruby 1.0.0-rc8, like ruby 2.4.4, GraalVM CE Native [x86_64-linux]

[chrisseaton]

I went looking for a document that explains our threading model, but I can't find one. We do actually have some major innovations in thread safety - look at our papers under the parallelism and concurrency section of https://rubybib.org. We're also the major contributors now to Concurrent Ruby.

This seems like a bug which @eregon will be able to fix, rather than some fundamental problem.

We're very interested in other thread stress tests and failures you find so please keep them coming if you find more.

We should also write a document on our threading and memory model, using work from those papers, and we'll compare it to JRuby.

[larskanis]

It's not as bad as all that. Two of the errors were due to a race condition in some Eventbox test cases, so that I completed my list of issues now.

So sorry for the noise and thank you for your awesome work on Ruby!

[eregon]

Thank you for the report, this case is a race condition when creating a new thread, I'll fix it soon.

TruffleRuby tries to follow the proposed Ruby Memory Model.
I also plan to add support for thread-safe built-in collections, Array and Hash, so semantics are similar to MRI, yet threads execute in parallel.
My general opinion is any low-level exception like this is a bug.

[eregon]

@larskanis I fixed the issue in 2c7190c.
It will be in the next release.

Documenting our threading and memory model would be nice, but I consider it out of scope of this issue, as the original bug is fixed.

[larskanis]

@eregon Thank you for fixing this! And thank you for linking to the proposed Ruby Memory Model. I wasn't aware of this document and the related discussion. But this is interesting and sure, I have my own opinions about it...

[eregon]

But this is interesting and sure, I have my own opinions about it...

@larskanis please share your thoughts on the Ruby tracker, feedback is welcome :)