Transformer step intermittently hangs forever

[jbeemster]

The transform step can occasionally hang and simply never exits; no errors are thrown and there is no indication in the EMR logs as to what is going wrong with the job step. The only solutions is to terminate the job and then restart.

Version: 0.3.1
Platform: Amazon EMR 5.9.0

[chuwy]

Am I right that this behavior observed only on persistent cluster?

[jbeemster]

Hi @chuwy - no this has been observed on both ephemeral and persistent. It does not seem related to long running processes as such.