AwsCustomResource: support JMESPath queries

[aripalo]

Allow filtering of Array<map> with AwsCustomResource's getData / getDataString methods.

Currently it seems that one can only access array items by their index (e.g. Items.0.Title.S), but that doesn't help when you can't trust the ordering of the array.

Use Case

Many AWS SDK methods return Array<map> structures in the response data.

One example is describeStacks method in CloudFormation service that returns the stack outputs as Array<map> structure.

I have a use case where I have to fetch values from stacks in different regions. I already have them available as CloudFormation stack outputs.

As a workaround I have implemented my own Lambda backed custom resource that manages this, but AwsCustomResource gets really close to providing that functionality which is frustrating 😅 Just to clarify that the AwsCustomResource is super cool thing already!

Proposed Solution

If using AWS APIs via AWS CLI, one can query these Array<map> structures with JMESPath compliant strings:

--query "Stacks[0].Outputs[?OutputKey=='Foo'].OutputValue" --output text

Above example from stackoverflow.com/a/41629516.

It would be nice if getData/getDataString supported similar filtering possibilities with JMESPath (or similar manner) so one could do something like this:

const outputs = new cr.AwsCustomResource(this, "MyStackOutputs", {
  onUpdate: { // will also be called for a CREATE event
    service: 'CloudFormation',
    action: 'describeStacks',
    parameters: {
      StackName: 'MyOutputStack',
    },
    physicalResourceId: Date.now().toString(), 
    region: "eu-west-1",
    outputPath: "Stacks[0].Outputs",
  }
});

outputs.getDataString("[?OutputKey=='Foo'].OutputValue");
                       ^^^^^^^^^^^^^^^^^^^

Other

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[aripalo]

One idea could be that to prevent breaking changes we'd introduce a new method: Maybe even getJmespath or something like that?

https://www.npmjs.com/package/jmespath would be the library to use there…

… but I've just started looking at the code that defines getData/getDataString, so definitely this is is more complex problem than I first imagined.

[aripalo]

Also I was looking to use AWS SDK SSM getParameters via AwsCustomResource & AwsSdkCall and it has the same problem as returned Parameters is Array<map>.

Basically my use case here is that I have many values in region A that need to be accessed in region B.

I guess I could work around it by defining multiple SSM parameters and multiple instances of AwsCustomResource (and specifying the target parameter as prop), but I also feel a bit uncomfortable with several custom resources in one stack.

[eladb]

Copy: @jogold

[jogold]

@aripalo This would require adding some sort of jmesPathExpression prop then we can do the filtering in the runtime code of the custom resource and return another flattened JSON as output (custom resources can only return name-value pairs). This cannot be solved in getData/getDataString which simply returns a path in the custom resource's output.

[aripalo]

Yeah, that seems to be the case. I ended up solving my use case with my own custom Lambda resource that uses Boto3 get_parameters_by_path with Recursive=True setting for SSM Parameters and then returns a a flat dictionary in the output so I can access any of the parameters with their path:

crossRegionParameters.output.getAttString("/foo/bar/1")

Might open source that construct later.

As far as I've looked around I ended up in the same conclusion that doing the jmespath filterin with AwsCustomResource is not doable in the current setup and for the use cases similar to mine it's just better to create own specific custom resource.

Closing this issue for now.