Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(stepfunctions): Support for JSON Path in maxConcurrency field of step function map step #20152

Closed
2 tasks
jstag711 opened this issue Apr 29, 2022 · 5 comments · Fixed by #20279
Closed
2 tasks
Assignees
Labels
@aws-cdk/aws-stepfunctions Related to AWS StepFunctions bug This issue is a bug. effort/small Small work item – less than a day of effort p1

Comments

@jstag711
Copy link

Describe the feature

I would like to use a Json path to specify the max concurrency for my map task in my step function. Presently you must give it a fixed integer.

Use Case

I have a use case where I would like to specify the max concurrency as input to my step function so that I can change that without re-deploying my step function definition.

Proposed Solution

Allow JSON path for max concurrency

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

1.152.0

Environment details (OS name and version, etc.)

MacOS 10.15.7

@jstag711 jstag711 added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Apr 29, 2022
@github-actions github-actions bot added the @aws-cdk/aws-lambda Related to AWS Lambda label Apr 29, 2022
@kaizencc kaizencc changed the title Support for JSON Path in maxConcurrency field of step function map step (stepfunctions): Support for JSON Path in maxConcurrency field of step function map step May 9, 2022
@github-actions github-actions bot added the @aws-cdk/aws-stepfunctions Related to AWS StepFunctions label May 9, 2022
@kaizencc kaizencc removed the @aws-cdk/aws-lambda Related to AWS Lambda label May 9, 2022
@kaizencc
Copy link
Contributor

kaizencc commented May 9, 2022

Hi @jstag711! Can you try using JsonPath.numberAt('$.Field')?

@kaizencc kaizencc added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed needs-triage This issue or PR still needs to be triaged. labels May 10, 2022
@jstag711
Copy link
Author

This is what I tried, it fails to synth:

Exception in thread "main" software.amazon.jsii.JsiiException: Validation failed with the following errors: maxConcurrency has to be a positive integer

Code:

software.amazon.awscdk.services.stepfunctions.Map batchRedrawTasks = new software.amazon.awscdk.services.stepfunctions.Map(this, "ConcurrentRedrawTasks",
                MapProps.builder()
                        .maxConcurrency(JsonPath.numberAt("$.maxConcurrency"))
                        .itemsPath(JsonPath.stringAt("$.batchParams"))
                        .parameters(Map.of(TASK_PARAMS + ".$", "$$.Map.Item.Value"))
                        .build());

@kaizencc
Copy link
Contributor

Ah. Looks like an aggressive synth time check that did not account for jsonPaths. I'll try to fix this today.

@kaizencc kaizencc added bug This issue is a bug. p1 effort/small Small work item – less than a day of effort and removed feature-request A feature should be added or improved. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. labels May 10, 2022
@jstag711
Copy link
Author

How should the numbers finally be expressed in the state machine definition? When I try manually editing it in the console to be this, it also complains: "MaxConcurrency": "$.maxConcurrency"

@mergify mergify bot closed this as completed in #20279 May 11, 2022
mergify bot pushed a commit that referenced this issue May 11, 2022
…20279)

Allows specifying `JsonPath.numberAt()` in `maxConcurrency` without synth-time errors and fixes #20152 

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

wphilipw pushed a commit to wphilipw/aws-cdk that referenced this issue May 23, 2022
…ws#20279)

Allows specifying `JsonPath.numberAt()` in `maxConcurrency` without synth-time errors and fixes aws#20152 

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-stepfunctions Related to AWS StepFunctions bug This issue is a bug. effort/small Small work item – less than a day of effort p1
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants