workflow step decorator to handle asynchronous communication with subsystem

[hanstrompert]

Problem

A common way to implement asynchronous communication with a substem in a workflow today uses two workflow steps, one step to prepare and submit the payload to the subsystem, and a following step with an input form that will wait for the subsystem to post back the results of the submitted job. This works just fine when the submitted job succeeded and the workflow can continue to execute the next steps in the workflow. In case the submitted job fails you are stuck, after the problem in the subsystem is fixed it should be possible to resubmit the job, but that is not possible because you are already in a step after the succeeded job submit step. And continuing the workflow is also not an option because the job in the subsystem has failed. This violates the WFO philosophy that states that it should be possible to retry every step until it succeeds and can continue with the next step until the workflow finishes.

Solution

Design a workflow step decorator that:

• combines the job submit, wait for the result, and validation of the result into one step
• allows the registration of a callback URL to be used by the subsystem to post the result back
• add a new process state to indicate that the process is waiting on a callback

[pboers1988]

Must Haves:

• Use the inputform generator to define the callback protocol
• Implement a configurable timer on the callback step
• This timer will only be used in the UI to enable or disable the retry button
• When retry has been triggered the old callback url must be disabled.
• Make it usable in the form of a @callback_step decorator