Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using null in AWS::Serverless::StateMachine in the ResultPath throws incorrect E0000 error #1657

Closed
dontirun opened this issue Aug 17, 2020 · 22 comments · Fixed by #2242
Closed
Labels
bug Something isn't working

Comments

@dontirun
Copy link
Contributor

StateMachines allow for a null value in the ResultPath Property to pass input directly to the output

Below is a minimal example of a template that is proper but flagged as invalid

AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31

Resources:
  rLambda:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: python3.6
      InlineCode: |
        def handler(event, context):
          print(event)
          return


  rTestMachine:
    Type: AWS::Serverless::StateMachine
    Properties:
      Definition: 
        StartAt: myState
        States:
          myState:
            Type: Task
            Resource:  !GetAtt rLambda.Arn
            ResultPath: null
            End: true
      Policies:
      - Statement:
        - Sid: Test
          Effect: Allow
          Action:
          - lambda:Invoke*
          Resource: !GetAtt rLambda.Arn

Adding quotes around null does not resolve the issue as the value of the property needs to evaluate to null. An exception for null checking should be added for this Parameter

@kddejong
Copy link
Contributor

Thanks for submitting this issue.

For my tracking this is the translated template. This gets translated to string which masks the null to the CloudFormation service. The cfn-lint ordering may be adjust to handle this situation correctly.

   "DefinitionString": {
          "Fn::Join": [
            "\n",
            [
              "{",
              "    \"StartAt\": \"myState\",",
              "    \"States\": {",
              "        \"myState\": {",
              "            \"End\": true,",
              "            \"Resource\": \"${definition_substitution_1}\",",
              "            \"ResultPath\": null,",
              "            \"Type\": \"Task\"",
              "        }",
              "    }",
              "}"
            ]
          ]
        },

@kddejong
Copy link
Contributor

If this remained as yaml or json (not as a string) and wasn't translated by SAM there would be a [/Resources/rTestMachine/Type/DefinitionString/States/myState/ResultPath] 'null' values are not allowed in templates error.

@sc-alscient
Copy link

sc-alscient commented Feb 4, 2021

Hi is there any workaround for this until the bug is resolved? Other than using json! Thanks.

I tried this but it doesn't stop it erroring.

Resources:
  StateMachine:
    Type: AWS::Serverless::StateMachine
    Metadata:
      cfn-lint:
        config:
          ignore_checks:
            - E0000

@PatMyron
Copy link
Contributor

PatMyron commented Feb 4, 2021

@sc-alscient
Copy link

Thanks for getting back to me so quickly. Does this apply for whole template as well? I tried the below but still have the same problem:

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: UI Cypress testing resources

Metadata:
  cfn-lint:
    config:
      ignore_checks:
        - E0000

@mndeveci
Copy link

mndeveci commented Jun 9, 2021

Hi,

With SAM v1.36.0 and SAM CLI v1.24.0 versions, we now support adding Metadata to the serverless resources.

Can you verify if that will help with your use case?

Thanks!

@nikgibbens
Copy link

Where are we at with this? It's 2022 and the problem still persists in CloudFormation with YAML.

@edmundcong
Copy link

Any ETA as to when this will be fixed? it's forcing our step after our map to handle our incoming input in a remarkably awkward way

@tianmarin
Copy link

Maybe i'm missing something, this issue is closed because there is no error thrown now, but yaml template with ResultPath: null won't create a state with a ResultPath equals to null, it will only omit the ResultPath parameter.

@kddejong
Copy link
Contributor

kddejong commented Nov 4, 2022

This is the template returned by running cfn-lint template.yaml --info

{
  "rTestMachine": {
    "Properties": {
      "DefinitionString": {
       "Fn::Join": [
        "\n",
        [
         "{",
         "    \"StartAt\": \"myState\",",
         "    \"States\": {",
         "        \"myState\": {",
         "            \"End\": true,",
         "            \"Resource\": \"${definition_substitution_1}\",",
         "            \"ResultPath\": null,",
         "            \"Type\": \"Task\"",
         "        }",
         "    }",
         "}"
        ]
       ]
      },
      "DefinitionSubstitutions": {
       "definition_substitution_1": {
        "Fn::GetAtt": [
         "rLambda",
         "Arn"
        ]
       }
      },
      "RoleArn": {
       "Fn::GetAtt": [
        "rTestMachineRole",
        "Arn"
       ]
      },
      "Tags": [
       {
        "Key": "stateMachine:createdBy",
        "Value": "SAM"
       }
      ]
     },
     "Type": "AWS::StepFunctions::StateMachine"
   }
}

@tianmarin
Copy link

Thank tou very much for your quick response!

Then it's a CloudFormation error?

On CF console when I see the template it is shown like this:

                SaveInfoToBucket:
                  Type: Task
                  ResultPath: null
                  Next: OtherStep
                  Retry:
                  - ErrorEquals:
                    - S3.SdkClientException
                    IntervalSeconds: 3
                    MaxAttempts: 3
                  Parameters:
                    Body.$: $
                    Bucket:
                      Ref: BucketId
                    Key.$: States.Format('{}_{}.json', $$.Execution.StartTime, $.field)
                  Resource: arn:aws:states:::aws-sdk:s3:putObject

But if i click View processed template:

              "                   \"SaveInfoToBucket\": {",
              "                        \"Next\": \"OtherStep\",",
              "                        \"Parameters\": {",
              "                            \"Body.$\": \"$\",",
              "                            \"Bucket\": \"${definition_substitution_12}\",",
              "                            \"Key.$\": \"States.Format('{}_{}.json', $$.Execution.StartTime, $.field)\"",
              "                        },",
              "                        \"Resource\": \"arn:aws:states:::aws-sdk:s3:putObject\",",
              "                        \"Retry\": [",
              "                            {",
              "                                \"ErrorEquals\": [",
              "                                    \"S3.SdkClientException\"",
              "                                ],",
              "                                \"IntervalSeconds\": 3,",
              "                                \"MaxAttempts\": 3",
              "                            }",
              "                        ],",
              "                        \"Type\": \"Task\"",
              "           

@kddejong
Copy link
Contributor

kddejong commented Nov 4, 2022

That is kind of baffling to me as the sam translator library from pypi will convert it to this:

"DefinitionString": {
     "Fn::Join": [
      "\n",
      [
       "{",
       "    \"StartAt\": \"myState\",",
       "    \"States\": {",
       "        \"SaveInfoToBucket\": {",
       "            \"Next\": \"OtherStep\",",
       "            \"Parameters\": {",
       "                \"Body.$\": \"$\",",
       "                \"Bucket\": \"${definition_substitution_1}\",",
       "                \"Key.$\": \"States.Format('{}_{}.json', $$.Execution.StartTime, $.field)\"",
       "            },",
       "            \"Resource\": \"arn:aws:states:::aws-sdk:s3:putObject\",",
       "            \"ResultPath\": null,",
       "            \"Retry\": [",
       "                {",
       "                    \"ErrorEquals\": [",
       "                        \"S3.SdkClientException\"",
       "                    ],",
       "                    \"IntervalSeconds\": 3,",
       "                    \"MaxAttempts\": 3",
       "                }",
       "            ],",
       "            \"Type\": \"Task\"",
       "        }",
       "    }",
       "}"
      ]
     ]
    },

They don't look similar this may be an issue unrelated to cfn-lint since we rely on SAM to transform the template appropriately.

@tianmarin
Copy link

@kddejong Thank you very much again for your quick response

After your response, I checked every tiny thing that might be causing this and voilà 🥳

My template uses two transformations:

---
AWSTemplateFormatVersion: 2010-09-09
Transform: 
  - 'AWS::LanguageExtensions'
  - 'AWS::Serverless-2016-10-31'

They are in the correct order as stated here:
Introducing new language extensions in AWS CloudFormation

Once I removed the AWS::LanguageExtensions (as it is not used in my AWS::Serverless::StateMachineresources), the transformation of SAM works perfectly:

              "     },",
              "     \"Resource\": \"arn:aws:states:::aws-sdk:s3:putObject\",",
              "     \"ResultPath\": null,",
              "     \"Retry\": [",
              "     {",

I hope this is useful for someone else.

@kddejong
Copy link
Contributor

kddejong commented Nov 4, 2022

that is useful and I wonder if its a bug in how the AWS::LanguageExtensions works.

@niokolra
Copy link

niokolra commented Mar 9, 2023

I now face the same issue when the cf is in YAML and the definition is JSON. I am unable to deploy my step function while ResultPath value is null

@dontirun
Copy link
Contributor Author

dontirun commented Mar 9, 2023

@niokolra I don't think the deployment error is related to cfn-lint.

CloudFormation will throw an error if it detects null values in YAML. You can get around that by switching from AWS::StepFunctions::StateMachine to AWS::Serverless::StateMachine and using the serverless transform in your CloudFormation template.

This gets around the issue because behind the scenes the Serverless Transform converts the YAML specified in theAWS::Serverless::StateMachine to stringified JSON before attempting to deploy the template

@dontirun
Copy link
Contributor Author

dontirun commented Mar 9, 2023

I misread the question, ignore my previous comment 🤦

@kddejong
Copy link
Contributor

kddejong commented Mar 9, 2023

@niokolra can you provide a few more details. Are you using the language transform? Are you using SAM? Or is it just a CloudFormation template with a step function defined in JSON

@niokolra
Copy link

niokolra commented Mar 9, 2023

@kddejong this is just a CloudFormation template with a step function defined in JSON. When I try to deploy it I receive the error

@kddejong
Copy link
Contributor

kddejong commented Mar 9, 2023

@niokolra do you get that error from the service or cfn-lint?

@MalikAtalla-AWS
Copy link
Contributor

Just for visibility: The underlying issue in the AWS::LanguageExtensions Transform has been fixed.

@mbwilding
Copy link

I misread the question, ignore my previous comment 🤦

Hey, you solved it for me, I swapped to the serverless version, thanks for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.