stepfunctions: associateWithParent should place id alongside existing input

[snowe2010]

Describe the feature

The associateWithParent flag should add a token to the existing payload when no input specified.

State input:

{
  "prop1": "value1",
  "props2-299": "...",
  "prop300": "value300" 
}

associateWithParent:

{
  "AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID": "idhere",
  "prop1": "value1",
  "props2-299": "...",
  "prop300": "value300" 
}

Use Case

There is a significant amount of extra states needed to pass the original input with the AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID id to a StartExecution call. This makes it difficult to keep state machines compartmentalized.

Imagine I have two state machines.
State Machine "Alpha" which is called by hundreds of services. It takes an object with several hundred fields.
State Machine "Beta" which calls Alpha.

With the current functionality you cannot use associateWithParent at all, because of the way that Paths work. You'd need to specify each and every field from the original input which would be impossible to maintain. Instead you have to modify Beta to have two extra Pass states. The first Pass creates a new payload with two separate nested fields, the original input and the executionId. Then the second takes and merges these top level fields and then filters the output.

Pass 1:

stateName = "Get the Execution ID"
parameters = mapOf(
    "json1" to mapOf("AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID" to JsonPath.getExecutionId()),
    "json2.$" to "$"
)

Pass 2:

parameters = mutableMapOf(
    "output.$" to "States.JsonMerge($.json1, $.json2, false)"
)
outputPath = "$.output"

I'm not sure why you would ever need the current functionality, which is to completely erase the existing payload and replace it with the AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID. It seems like it would be impossible to actually pass any data to your State Machine. I can understand wanting to filter the output and place it directly alongside the execution id. No other CDK Step Function construct works this way, where if the input is not specified then the input is entirely erased and it is very confusing when you activate associateWithParent and suddenly your input is erased and replaced with a token you would expect to appear alongside your input.

Proposed Solution

If an input (TaskInput) is not specified then it should just place the AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID directly alongside the existing task input. The task input should not be nested below another key.

Turning on associateWithParent should do this to the input:

{
  "param1": "value1"
}

to

{
  "AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID": "idhere",
  "param1": "value1"
}

NOT this:

{
  "AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID": "idhere"
}

If a TaskInput is specified in the input property then that should follow the existing documentation around adding fields to the input.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

software.amazon.awscdk:aws-cdk-lib:2.158.0

Environment details (OS name and version, etc.)

macOS Sonoma 14.7

[khushail]

Hi @snowe2010 , thanks for reaching out. Although I am not very familiar with usage of associateWithParent but I found a mention in cdk docs that by turning associateWithParent true, this will add the payload AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID.$: $$.Execution.Id to the inputproperty for you, which will pass the execution ID from the context object to the execution input. It requires input to be an object or not be set at all.
I
Ref -

aws-cdk/packages/aws-cdk-lib/aws-stepfunctions-tasks/README.md

Line 1777 in 8208774

You can utilize [Associate Workflow Executions](https://docs.aws.amazon.com/step-functions/latest/dg/concepts-nested-workflows.html#nested-execution-startid)

aws-cdk/packages/aws-cdk-lib/aws-stepfunctions-tasks/README.md

Line 1789 in 8208774

This will add the payload `AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID.$: $$.Execution.Id` to the

Also -
https://docs.aws.amazon.com/step-functions/latest/dg/concepts-nested-workflows.html#nested-execution-startid

https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_stepfunctions_tasks.StepFunctionsStartExecution.html#associatewithparent

Pass the execution ID from the context object to the execution input.

This allows the Step Functions UI to link child executions from parent executions, making it easier to trace execution flow across state machines.

If you set this property to true, the input property must be an object (provided by sfn.TaskInput.fromObject) or omitted entirely.

so in case where input is not specified, the issue arises with overriding the flag value. In this case, your proposal makes sense but since this could be a breaking change, I would leave it upto team to decide on this change.

Requesting team to share their insights on this FR and provide comments if this is something they think should be changed or can be improved.

[gracelu0]

Hi @snowe2010 , could you provide an example CDK app for us to reproduce this issue? How are you defining/passing the input currently? From my understanding the associateWithParent flag was implemented such that it will only append the execution ID if the input is defined using sfn.TaskInput.fromObject. I do understand the expectation that it should not overwrite the input, this may be something we need to make clearer in the documentation.