Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: create Java cfn-response equivalent (#558) #560

Merged
merged 7 commits into from
Oct 26, 2021

Conversation

bdkosher
Copy link
Contributor

@bdkosher bdkosher commented Sep 28, 2021

Introduce Java implementation of the cfn-response module based on NodeJS design.

Issue #, if available:

Addresses #558

Description of changes:

  • Add new powertools-cloudformation Maven module to house the sole Java class, CloudFormationResponse.
  • Include one new parent POM dependency on software.amazon.awssdk:http-client-spi:${aws.sdk.version}.
  • Add Unit Test class with 100% code coverage.

Checklist

Breaking change checklist

None. New code.

RFC issue #:

  • Migration process documented
  • Implement warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@bdkosher
Copy link
Contributor Author

Is the SpotBugs / codecheck failure a false positive?

The logs do not show any violations for the impacted module:
/home/runner/work/aws-lambda-powertools-java/aws-lambda-powertools-java/powertools-cloudformation/target/spotbugsXml.xml has 0 violations

@msailes
Copy link
Contributor

msailes commented Sep 30, 2021

A couple of questions from me.

  • Do you think it would be useful to include a builder for ResponseBody this would give customers a similar feel to using the v2 SDK and other AWS Java libraries?
  • Can you add documentation for this module so it appears on the website - https://awslabs.github.io/aws-lambda-powertools-java/

@bdkosher
Copy link
Contributor Author

  1. ResponseBody is currently more of an internal object for producing the desired JSON payload. We could directly expose that object for maximal control and have that be passed to a singular "send" method instead of having various polymorphic "send"s with different parameters. Alternatively, we could make the overall class follow the builder pattern, culminating in a "build" that would actually perform the "send". I do like the idea of having a builder here one way or another. I'm not particularly fond that I cannot specify a noSend arg without having to explicitly specify a physicalResourceId arg as well.
  2. Certainly! I totally didn't notice the docs folder earlier.

Another design issue for your consideration: Right now there's a single ObjectMapper that arguably does double-duty: format the overall JSON in the HTTP request and format the Object responseData that gets included in that payload.

For my own application, e.g. I would ideally like to pass a software.amazon.awssdk.services.chime.model.CreateAppInstanceResponse as the "responseData", but this class does not serialize nicely. I could pass in my own ObjectMapper that would support serializing this type, but then that might interfere with overall JSON serialization (which is why I've currently made the constructor that accepts the ObjectMapper arg protected). Some ideas:

  • Instead of a generic Object arg, use Map<String, Object> or possibly Map<String, String> to better mimic the "list of name value pairs" described in the JS/Python versions.
  • Allow a custom ObjectMapper to be provided on init, but keep it separate from the generic ObjectMapper used to produce the overall JSON (might be a bit tricky to implement)
  • Or just leave things the way they are.

@pankajagrawal16
Copy link
Contributor

Is the SpotBugs / codecheck failure a false positive?

The logs do not show any violations for the impacted module: /home/runner/work/aws-lambda-powertools-java/aws-lambda-powertools-java/powertools-cloudformation/target/spotbugsXml.xml has 0 violations

Yeah I have been trying to figure this one out. It looks like it fails on PRs where author donot have access to repo. I need to find some time to troubleshoot this further.

*
* @param client HTTP client to use for sending requests; cannot be null
*/
public CloudFormationResponse(SdkHttpClient client) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be good to provide an implementation where user dont need to care about passing http client as well and module can take care of setting it up.

From Custom resource perspective, I believe it will provide better developer experience.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. The JS and Python versions embed the client. We'd just need to

  1. Pick a particular client impl (Apache?) and
  2. Include it as a dependency to the project.

@pankajagrawal16
Copy link
Contributor

pankajagrawal16 commented Oct 1, 2021

@bdkosher Firstly, Thanks for opening the issue and PR.

I am thinking out loud here but will it be better builder experience if we provide abstract implementation like this:

public abstract class CloudformationCustomResource {
    
    public Object handleRequest(CloudFormationCustomResourceEvent input, Context context) {
        Response response = null;

        switch (input.getRequestType()) {
            case "CREATE":
                response = create(input, context);
                buildAndSendCfnResponse(response, input, context);
                break;
            case "UPDATE":
                response = update(input, context);
                buildAndSendCfnResponse(response, input, context);
                break;
            // and so on
        }

        return response;
    }

    protected void buildAndSendCfnResponse(Response response, CloudFormationCustomResourceEvent input, Context context) {
        // Map Response to Response Body
        // Send response
    }

    public abstract Response create(CloudFormationCustomResourceEvent input, Context context);

    public abstract Response update(CloudFormationCustomResourceEvent input, Context context);

    public abstract Response delete(CloudFormationCustomResourceEvent input, Context context);
}

Then from builder perspective, we can control alot of things, for example, Response object can have all the builder goodness and be very specific to what we expect users to provide us. It can be a good place to capture mapper as which they would want us to use to serialize response data.

Also then builders wont need to know too much details about how to handle custom resource events etc. From their perspective, they just need to do their stuff for create, update and delete and provide us a response object.

What do you think? I believe this is closer to the implementation we have for python one.

Builder experience then looks something like this:

public class CustomResourceHandler extends CloudformationCustomResource {


    @Override
    public Response create(CloudFormationCustomResourceEvent input, Context context) {
        return null;
    }

    @Override
    public Response update(CloudFormationCustomResourceEvent input, Context context) {
        return null;
    }

    @Override
    public Response delete(CloudFormationCustomResourceEvent input, Context context) {
        return null;
    }
}

@bdkosher
Copy link
Contributor Author

bdkosher commented Oct 1, 2021

Seems like a good idea; as far as I know, CloudFormationResponse only gets used in the context of a handler that does some sort of one-time stack setup action based on the event type. So if I understand you correctly, we'd add two more classes:

  1. An abstract base handler class that would allow subclasses to just "do their stuff for create, update and delete and provide us a response object"
    1. Should we have the create/update/delete methods call send() with ResponseStatus = SUCCESS, Response = null by default? Subclasses would only need to override the methods they care about.
    2. What's the proper way to handle the situation when communication to the custom resource fails? Invoke some other method that subclasses could override if they wanted to?
    3. Should this class itself implement RequestHandler?
  2. A custom Response type to model the response data included in the send payload, plus a nested Builder for that type.

I'm debating between having Response hold name/value pairs vs. essentially be a wrapper for some java.lang.Object with option to customize its ObjectMapper. I know for my application's case, it's more convenient to manually convert my response object to a Map than configuring a custom ObjectMapper.

void sendResponse(event, context, status, CreateAppInstanceResponse chimeResponse) {
    // easier to do this than configure a mapper to serialize artifact object type
    Map<String, String> data = Map.of("AppInstanceArn", chimeResponse.appInstanceArn());
    new CloudFormationResponse(ApacheHttpClient.create()).send(event, context, status, data);
}

@pankajagrawal16
Copy link
Contributor

Should we have the create/update/delete methods call send() with ResponseStatus = SUCCESS, Response = null by default? Subclasses would only need to override the methods they care about.

Idea was to basically let users return a custom response object which we provide with builders etc. Users can decide based on the processing what will be the status etc. But basically we accept most of the parameters (Some optional probably) which users would want to send in response. This will include a custom object as well as part of response. The response object can then as well provide a method to accept custom object mapper which they would probably want to use when serializing response object. Builder dont need to then worry about sending the repsonse itself. That will be taken care by the handler which we implement as part of CloudformationCustomResource . Makes sense ?

@pankajagrawal16
Copy link
Contributor

What's the proper way to handle the situation when communication to the custom resource fails? Invoke some other method that subclasses could override if they wanted to?

It might be an idea to give an optional hook which is called in case send response invocation fails. Although I wonder what would user do in that case ? But still it might be a good idea to provide that flexibility .

@pankajagrawal16
Copy link
Contributor

Should this class itself implement RequestHandler?

That was my thought yes. So Builder just need to provide implementation of three methods and return the response object.

@pankajagrawal16
Copy link
Contributor

I'm debating between having Response hold name/value pairs vs. essentially be a wrapper for some java.lang.Object with option to customize its ObjectMapper. I know for my application's case, it's more convenient to manually convert my response object to a Map than configuring a custom ObjectMapper.

I believe it can accept any object, which could be a map as well if they want it to be. We can document that by default it will be serialized using default instance of object mapper, but if they want to use specific object mapper configuration, they can provide one as part of the custom Response object.

@pankajagrawal16
Copy link
Contributor

Is the SpotBugs / codecheck failure a false positive?
The logs do not show any violations for the impacted module: /home/runner/work/aws-lambda-powertools-java/aws-lambda-powertools-java/powertools-cloudformation/target/spotbugsXml.xml has 0 violations

Yeah I have been trying to figure this one out. It looks like it fails on PRs where author donot have access to repo. I need to find some time to troubleshoot this further.

Try rebasing with master and that should now fix spotbugs check issue for now.

@bdkosher
Copy link
Contributor Author

bdkosher commented Oct 6, 2021

Pushed some changes for your consideration.

I added the abstract resource handler and Response classes (with Builder) as we discussed, as well as a ResponseException (I wasn't feeling particularly creative with the name so I'm open to alternatives). The CloudFormationResponse is still there. It had been named to mimic the "cfn-response" module, but now with these news classes I'm thinking we should rename it to ResponseSender or something along those lines.

ResponseException is used to differentiate between an Exception occurring while constructing a response (ResponseException) vs. sending a response (IOException). Constructing the response can fail in two ways:

  1. There's a RuntimeException thrown by the create/update/delete method of the handler
  2. There's some RuntimeException/JacksonException thrown while attempting to merge the ResponseBody wrapper object with the Response object that ends up as the "Data" property.

I'm not fond of the the nested try/catches being used to handle these two failure scenarios--maybe there's a way to refactor CloudFormationResponse to just deal with "response sending" concerns and put the "response construction concerns" elsewhere that could be handled by a single try/catch.

If there's an exception while sending the response, I'm currently just logging it and invoking the empty-by-default onSendFailure method on the handler--there's not much you can really do at that point, as far as I know.

A final note, I'm still requiring the SdkHttpClient to be injected. I figured we could review these changes first and later figure out which HTTP client impl we wanted to use by default.

try {
response = getResponse(event, context);
LOG.debug("Preparing to send response {} to {}.", response, responseUrl);
client.send(event, context, ResponseStatus.SUCCESS, response);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be its not rite to assume that user will always like to send status as Success ? Probably Response object should accept status as a field too and defaulting it to success ?

Copy link
Contributor Author

@bdkosher bdkosher Oct 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this. We could also even allow the physical resource ID and noEcho flag to be specified in the Response. Otherwise, with the way the handler is set up, there's no way to customize these values outside of invoking CloudFormationResponse::send directly from a custom handler.

Will still send success if the entire response is null; returning null from the create/update/delete methods is basically saying "I don't care about these type of of events".

* <p>
* This class is thread-safe provided the SdkHttpClient instance used is also thread-safe.
*/
public class CloudFormationResponse {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the implmentation here should now be internal ? and not be public as users only need to care about providing implementation for the abstract class ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with that. I had left it public in case anyone wanted to write their own custom handler, but given the nature of this code, I can't come up with a good reason why anyone would actually do that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still believe it's better to close it down. See it's easier to open it up based on community feedback and hearing use cases around it later than starting with an open one. Then its impossible to close it ever.

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

public class AbstractCustomResourceHandlerTest {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would the test be simpler to just verify and argument captor on mocked http client calls and arguments ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Verifying the call to SdkHttpClient::prepareRequest is tricky because the HttpExecuteRequest arg doesn't expose readily-usable methods for validating things, but using Mockito to verify calls to create/update/delete on the handler itself would be more straightforward that manually counting invocations.

@pankajagrawal16
Copy link
Contributor

Pushed some changes for your consideration.

I added the abstract resource handler and Response classes (with Builder) as we discussed, as well as a ResponseException (I wasn't feeling particularly creative with the name so I'm open to alternatives). The CloudFormationResponse is still there. It had been named to mimic the "cfn-response" module, but now with these news classes I'm thinking we should rename it to ResponseSender or something along those lines.

ResponseException is used to differentiate between an Exception occurring while constructing a response (ResponseException) vs. sending a response (IOException). Constructing the response can fail in two ways:

  1. There's a RuntimeException thrown by the create/update/delete method of the handler
  2. There's some RuntimeException/JacksonException thrown while attempting to merge the ResponseBody wrapper object with the Response object that ends up as the "Data" property.

I'm not fond of the the nested try/catches being used to handle these two failure scenarios--maybe there's a way to refactor CloudFormationResponse to just deal with "response sending" concerns and put the "response construction concerns" elsewhere that could be handled by a single try/catch.

If there's an exception while sending the response, I'm currently just logging it and invoking the empty-by-default onSendFailure method on the handler--there's not much you can really do at that point, as far as I know.

A final note, I'm still requiring the SdkHttpClient to be injected. I figured we could review these changes first and later figure out which HTTP client impl we wanted to use by default.

This is great. Already starting to look nice. Left few suggestions, see if you agree with them.

@bdkosher
Copy link
Contributor Author

Pushed another commit taking most of your feedback into consideration.

  • I left CloudFormationResource public for the time being. Since other language modules effectively provide that object alone, I was hesitant to make it inaccessible for Java classes.
  • The abstract Handler class still requires an SdkHttpClient arg at present. Still unsure about adding a default impl and dependency.
  • No documentation updates included as of yet. I plan to add them to the PR once we finalize the APIs.

@pankajagrawal16
Copy link
Contributor

pankajagrawal16 commented Oct 14, 2021

Pushed another commit taking most of your feedback into consideration.

  • I left CloudFormationResource public for the time being. Since other language modules effectively provide that object alone, I was hesitant to make it inaccessible for Java classes.
  • The abstract Handler class still requires an SdkHttpClient arg at present. Still unsure about adding a default impl and dependency.
  • No documentation updates included as of yet. I plan to add them to the PR once we finalize the APIs.

I think everything looks much better. This is seems to shape up real nice now.

I still believe it's better to close CloudFormationResource down. See it's easier to open it up based on community feedback and hearing use cases around it later than starting with an open one. Then it's impossible to close it ever.

For Default client, I think especially for the use case of Custom resource, Cold start is not a big problem, so if I have to choose taking an extra dep over better builder experience and ease of use, I would say latter wins :)

@pankajagrawal16
Copy link
Contributor

In addition, I believe you also need to do changes here https://github.com/awslabs/aws-lambda-powertools-java/blob/master/.github/workflows/build.yml and few other files in .github folder for automation to work correctly for this module.

@bdkosher
Copy link
Contributor Author

I still believe it's better to close CloudFormationResource down. See it's easier to open it up based on community feedback and hearing use cases around it later than starting with an open one. Then it's impossible to close it ever.

Makes sense. Will update it to be package-private.

For Default client, I think especially for the use case of Custom resource, Cold start is not a big problem, so if I have to choose taking an extra dep over better builder experience and ease of use, I would say latter wins :)

True. OK with "software.amazon.awssdk:url-connection-client" as the default client? Looks to be the smallest client size-wise.

you also need to do changes here https://github.com/awslabs/aws-lambda-powertools-java/blob/master/.github/workflows/build.yml

OK. Looks like that file and spotbugs.yml.

Also, do you see value in making 3 separate handler sub-classes each specific to one operation, e.g. abstract class CreateCustomResourceHandler extends AbstractCustomResourceHandler would implement the update and delete methods to return null; the developer would only need to provide the create method impl.

@pankajagrawal16
Copy link
Contributor

pankajagrawal16 commented Oct 15, 2021

Also, do you see value in making 3 separate handler sub-classes each specific to one operation, e.g. abstract class CreateCustomResourceHandler extends AbstractCustomResourceHandler would implement the update and delete methods to return null; the developer would only need to provide the create method impl.

I would avoid doing this, One reason is to make it visible to customer that there are 3 parts to the puzzle of implementing a custom resource, so that its rite there for them to decide if they want to do anything on create or update or delete or not. Other is, Even though it will ease use and make code cleaner for ones who dont need it and know about it, I would avoid adding it until i see community ask as of now.

@pankajagrawal16
Copy link
Contributor

True. OK with "software.amazon.awssdk:url-connection-client" as the default client? Looks to be the smallest client size-wise.

Sounds good

@bdkosher
Copy link
Contributor Author

Pushed another commit with the .github config, default SdkClient, lowered CloudFormationResource visibility, and a spotbugs fix. Provided things look good to you and the automated checks pass, I'll start the documentation.

@pankajagrawal16
Copy link
Contributor

Pushed another commit with the .github config, default SdkClient, lowered CloudFormationResource visibility, and a spotbugs fix. Provided things look good to you and the automated checks pass, I'll start the documentation.

Looks good to me. Probably we can start with documentation :)

@pankajagrawal16
Copy link
Contributor

Missing entry of docs here https://github.com/awslabs/aws-lambda-powertools-java/blob/master/mkdocs.yml#L15

@pankajagrawal16
Copy link
Contributor

Looked through the docs and looks good.

@bdkosher
Copy link
Contributor Author

Thanks for all the feedback. I'll push this missing commits from master to remove the pom.xml conflict.

Joe Wolf added 6 commits October 22, 2021 11:30
Introduces Java implementation of cfn-response module based on NodeJS design.
Put the sole class in a separate powertools-cloudformation Maven module.
100% code coverage

Addresses aws-powertools#558
Provides abstract methods for generating the Response to be sent,
represented as its own type. Use Objects::nonNull instead of custom
method.

Addresses aws-powertools#558
…r itself

Instead of method args to various CloudFormationResponse::send methods, reducing the number of polymorphic send
methods to two: one with a Response arg and one without.

Rename ResponseException to CustomResourceResponseException.

AbstractCustomResourceHandlerTest simplifications.
- Include powertools-cloudformation in .github config
- Address mutatable ObjectMapper spotbugs finding
- JavaDoc cleanup
Copy link
Contributor

@pankajagrawal16 pankajagrawal16 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to go. Thanks for all the efforts @bdkosher 🙌

@pankajagrawal16 pankajagrawal16 changed the title Create Java cfn-response equivalent (#558) feat: create Java cfn-response equivalent (#558) Oct 24, 2021
@pankajagrawal16 pankajagrawal16 merged commit 35ad5f6 into aws-powertools:master Oct 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants