Isue115/inline and distributed maps #116

massfords · 2023-01-05T03:23:21Z

Added two test cases for the new map features.

The schema types could be better.

BREAKING CHANGE: updated the json path error

# Conflicts: # package-lock.json # src/checks/json-schema-errors.ts # src/validator.ts

tests: added new test for asserting error messages chore: updated tsconfig to keep js files out of src

# Conflicts: # package-lock.json # package.json # src/__tests__/error-reporting.test.ts # src/checks/json-schema-errors.ts # src/schemas/paths.json # src/validator.ts

# Conflicts: # src/checks/json-schema-errors.ts

massfords · 2023-01-05T03:23:44Z

src/__tests__/definitions/valid-map-distributed.asl.json

@@ -0,0 +1,48 @@
+{
+  "Comment": "Using Map state in Distributed mode",


the two example files added were taken from the AWS docs

massfords · 2023-01-05T03:24:37Z

src/checks/json-schema-errors.ts

+        }))
+        .filter((v: StateMachineError,i,a)=>a.findIndex(v2=>(v2.schemaError.schemaPath===v.schemaError?.schemaPath && v2.schemaError.instancePath===v.schemaError?.instancePath))===i);
 }


unrelated, but I noticed some duplicate error messages in an existing test after this change so this will remove any dupes before returning the errors

could you add some new lines to make it easier to read? And maybe add a comment to explain it's for error deduplication

massfords · 2023-01-05T03:25:27Z

src/schemas/map.json

+    "ItemReader": {
+      "$comment": "need to improve this type",
+      "type": "object"
+    },


I believe the fields for this are defined in the spec and don't vary. Easy enough to update to define more here.

massfords · 2023-01-05T03:27:04Z

src/schemas/map.json

    "ResultSelector": {
      "$ref": "paths.json#/definitions/asl_payload_template"
    },
+    "ResultWriter": {


need better typxingfor the ResultWriter as well. I think these fields are for the s3 bucket to write to.

could you add a $comment, like you did for ItemReader?

massfords · 2023-01-05T03:32:27Z

@ChristopheBougere Sorry for the failing build. I think it's a commit message issue. I made some quick changes to the schemas to support the new types

I haven't read through all of the new Map features yet. The examples I've added pass, but the map schema type still needs some work to avoid accepting definitions with both new and old properties.

ChristopheBougere

Thanks for working on this and for the comments @massfords!

A few comments, let me know what you think

FYI you can run a git rebase -i to rewrite your commit messages (or squash them). Otherwise, I'll squash them in github when merging the PR ;)

Also, is it ready to merge or do you want to work more on the ItemReader and ResultWriter before?

ChristopheBougere · 2023-01-05T17:42:20Z

src/schemas/base-state-machine.json

@@ -0,0 +1,23 @@
+{


Could you update the state-machine.json schema so it extends this base to avoid duplication?
If I understand well, it is the same without Version and TimeoutSeconds properties?
Also, I think we could use this base in the Branches properties of the parallel.json schema instead of using state-machine.json?

I recall trying this and hitting issues with additionalProperties. I think it's possible to get right with JSON Schema + AJV, I just didn't have the time to work through the issue. My approach was to have the base as its own file and then have state-machine.json reference it with an $allOf and then add its own properties.

ChristopheBougere · 2023-01-05T17:43:46Z

src/schemas/map.json

    "ResultSelector": {
      "$ref": "paths.json#/definitions/asl_payload_template"
    },
+    "ResultWriter": {


could you add a $comment, like you did for ItemReader?

ChristopheBougere · 2023-01-05T17:46:26Z

src/checks/json-schema-errors.ts

+        }))
+        .filter((v: StateMachineError,i,a)=>a.findIndex(v2=>(v2.schemaError.schemaPath===v.schemaError?.schemaPath && v2.schemaError.instancePath===v.schemaError?.instancePath))===i);
 }


could you add some new lines to make it easier to read? And maybe add a comment to explain it's for error deduplication

massfords · 2023-01-06T12:57:00Z

I'll have some time this weekend to clean up the PR and address the comments.

massfords · 2023-01-06T12:59:03Z

sorry, didn't mean to close the PR :(

massfords · 2023-01-12T20:36:59Z

@ChristopheBougere I've made a couple of changes to address the comments.

Please do squash my commits in the merge. I didn't realize how out of date my branch was.

github-actions · 2023-01-13T06:45:10Z

🎉 This PR is included in version 3.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

ChristopheBougere and others added 15 commits July 5, 2022 18:32

chore: rewrite to typescript

c09cc57

BREAKING CHANGE: updated the json path error

fix: typescript review (remove any)

24ad137

feat: integrate asl path validator (#2)

9611d48

feat(paths): update to latest path validator

ad98c39

Merge branch 'ts'

0fe8e69

# Conflicts: # package-lock.json # src/checks/json-schema-errors.ts # src/validator.ts

chore: resolve conflicts

d2a54d7

chore: merge cleanup

f7d5df6

refactor: extract arn formatter and added tests

efa7f44

fix: restored support for array

8e209c3

fix: restored support for cloud formation functions for resources

f8c08bb

feat: better filtering of error messages

38cfdc2

tests: added new test for asserting error messages chore: updated tsconfig to keep js files out of src

feat: initial support for new map fields

4da0628

Merge remote-tracking branch 'upstream/main'

83764c2

# Conflicts: # package-lock.json # package.json # src/__tests__/error-reporting.test.ts # src/checks/json-schema-errors.ts # src/schemas/paths.json # src/validator.ts

Merge branch 'main' into isue115/inline_and_distributed_maps

5c9cfa2

# Conflicts: # src/checks/json-schema-errors.ts

chore: updated branch

282db75

massfords commented Jan 5, 2023

View reviewed changes

ChristopheBougere linked an issue Jan 5, 2023 that may be closed by this pull request

Add support for new Distributed Map in Step Functions Definition #115

Closed

ChristopheBougere requested changes Jan 5, 2023

View reviewed changes

massfords closed this Jan 6, 2023

massfords reopened this Jan 6, 2023

chore: comments and formatting

f79fedd

ChristopheBougere merged commit 74772f7 into ChristopheBougere:main Jan 13, 2023

ChristopheBougere added a commit that referenced this pull request Jan 13, 2023

feat: inline and distributed maps (#116)

da9b2da

github-actions bot added the released label Jan 13, 2023

ChristopheBougere mentioned this pull request Jan 13, 2023

Add support for new Distributed Map in Step Functions Definition #115

Closed

massfords deleted the isue115/inline_and_distributed_maps branch February 15, 2023 01:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Isue115/inline and distributed maps #116

Isue115/inline and distributed maps #116

massfords commented Jan 5, 2023

massfords Jan 5, 2023

massfords Jan 5, 2023

ChristopheBougere Jan 5, 2023

massfords Jan 5, 2023

massfords Jan 5, 2023

ChristopheBougere Jan 5, 2023

massfords commented Jan 5, 2023

ChristopheBougere left a comment

ChristopheBougere Jan 5, 2023

massfords Jan 12, 2023

ChristopheBougere Jan 5, 2023

ChristopheBougere Jan 5, 2023

massfords commented Jan 6, 2023

massfords commented Jan 6, 2023

massfords commented Jan 12, 2023

github-actions bot commented Jan 13, 2023

		@@ -0,0 +1,48 @@
		{
		"Comment": "Using Map state in Distributed mode",

Isue115/inline and distributed maps #116

Isue115/inline and distributed maps #116

Conversation

massfords commented Jan 5, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

massfords commented Jan 5, 2023

ChristopheBougere left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

massfords commented Jan 6, 2023

massfords commented Jan 6, 2023

massfords commented Jan 12, 2023

github-actions bot commented Jan 13, 2023