You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hey. Thanks for publicizing this great piece of software! We are happily using embulk-output-orc in our production pipeline.
We found an incompatibility issue with Embulk >0.8 which enabled "Page Scattering"[release note] by default.
It seems the ScatterExecutor instantiates multiple instances of the plugin and end up running orc.Writer in parallel, while the default orc.MemoryManagerImpl is not thread-safe [code].
With Embulk >0.8, we get a lot of "Wrong Owner Thread" warnings [code], and random crashes when finalizing the write.
If you are open to contributions, we are happy to collaborate on a fix. We have a tentative fix which seems to be working on our prod, which we'll open a PR shortly.
(突然の Issue 失礼いたします。日本語のやりとりを希望される場合は、お申し付けください。)
The text was updated successfully, but these errors were encountered:
Hey. Thanks for publicizing this great piece of software! We are happily using
embulk-output-orc
in our production pipeline.We found an incompatibility issue with Embulk >0.8 which enabled "Page Scattering"[release note] by default.
It seems the ScatterExecutor instantiates multiple instances of the plugin and end up running
orc.Writer
in parallel, while the defaultorc.MemoryManagerImpl
is not thread-safe [code].With Embulk >0.8, we get a lot of "Wrong Owner Thread" warnings [code], and random crashes when finalizing the write.
If you are open to contributions, we are happy to collaborate on a fix. We have a tentative fix which seems to be working on our prod, which we'll open a PR shortly.
(突然の Issue 失礼いたします。日本語のやりとりを希望される場合は、お申し付けください。)
The text was updated successfully, but these errors were encountered: