Event Handler: ALB returns 502 for CORS Preflight configuration in ALBResolver

[francispeabody]

What were you searching in the docs?

This is either a bug or a documentation issue. I am looking at this section of the documentation. The way I read the documentation is that if cors_config is provided, powertools will automatically handle preflight 'OPTIONS' requests. In other words, you do not need to write and code to handle options requests as long as you provide a cors_config in the ALBResolver constructor. This does not appear to be the case. When fronting a lambda with an ALB, the preflight request fails, even if cors_config is provided, unless the options request is explicitly handled in the lambda code.

Is this related to an existing documentation section?

https://awslabs.github.io/aws-lambda-powertools-python/latest/core/event_handler/api_gateway/#pre-flight

How can we improve?

Clarification of the documentation for preflight requests for ALBs specifically. Also a code example of how to handle preflight requests for a lambda fronted by and ALB would be nice. Thanks!

Got a suggestion in mind?

No response

Acknowledgment

• I understand the final update might be different from my proposed suggestion, or refused.

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.

[heitorlessa]

hey @francispeabody, thanks for helping us improve our docs! I've just created a test and we're returning the CORS headers as expected - could you provide a sample or something we could try to reproduce?

We can deploy an ALB + Function to try later once we have more details.

Thank you!

[francispeabody]

Hi @heitorlessa I appreciate your quick response. Your test is basically the same as my code. The response you get is what I get as well. When I test my lambda is the lambda console, it successfully returns the preflight response, with the correct headers. I used the same event sent my my ALB to test in the lambda console (captured from the logs) to test via the lambda console and my function performed as expected. Here is the even used to test:

  "requestContext": {
    "elb": {
      "targetGroupArn": "arn:aws:elasticloadbalancing:us-east-1:207015420515:targetgroup/lendinguxCustomersDevTg/54603b56c7e4f9da"
    }
  },
  "httpMethod": "OPTIONS",
  "path": "/customers/008100000000000000000081691404/business",
  "queryStringParameters": {},
  "headers": {
    "accept": "*/*",
    "accept-encoding": "gzip, deflate, br",
    "connection": "keep-alive",
    "host": "sbblendinguxorchestrationlayerdev-lambda-us-east-1.clouddqt.capitalone.com",
    "postman-token": "c20edc33-1f1d-4386-bfb8-74d7ff4b97f1",
    "user-agent": "PostmanRuntime/7.29.0",
    "x-amzn-trace-id": "Root=1-629fa9ac-25cdcbe359b55649538baf26",
    "x-forwarded-for": "10.74.177.54",
    "x-forwarded-port": "443",
    "x-forwarded-proto": "https"
  },
  "body": "",
  "isBase64Encoded": false
}

This is the response from the console:

  "statusCode": 204,
  "headers": {
    "Access-Control-Allow-Origin": "*",
    "Access-Control-Allow-Headers": "Authorization,Content-Type,X-Amz-Date,X-Amz-Security-Token,X-Api-Key",
    "Access-Control-Max-Age": "10000",
    "Access-Control-Allow-Methods": "GET,OPTIONS"
  },
  "body": null,
  "isBase64Encoded": false
}

As you can see everything appears to be normal. When I make the call to my ALB, I can see my lambda is triggered from the logs and the response that i log out to cloud watch looks like the response above, however the alb still responds with a 502 bad gateway. I also see that the Lambda User Error metric for the lambda target group spikes when I get the 502 responses from the ALB.

Thanks again for the quick response. If you have to ability to test preflight request to a lambda from an ALB, I think that would be the best way to confirm if this is indeed an issue

[heitorlessa]

great thank you for the extra details @francispeabody. That could be likely a bug on our side. I remember ALB being picky about headers and values prior to Lambda support, so maybe statusCode must be 200, body can't be null - need to explore this more.

I'll setup a deployment and experiment some adjustments to reproduce and fix - we can convert it to a bug report.

[heitorlessa]

Reproduced! @francispeabody, it's our bug. Changing the labels to bug - repro here: https://github.com/heitorlessa/pt-1248

[heitorlessa]

@francispeabody here's a workaround for now while I find which header is causing ALB to panic: heitorlessa/pt-1248@67cc63e

[heitorlessa]

Found the issue - ALB apparently faults if the body is null - need to check RFC too. I'm gonna raise a ticket to the ALB team but in the meantime, we'll make a bug fix release. I just wanna make sure API Gateway doesn't have side effects with a empty bull (not null) before making this change.

[heitorlessa]

Confirmed, it's just ALB. CORS Specification also doesn't dictate whether the response body should be present, so it's more of a serialization issue in ALB - we'll change on our side for now and inform the ALB team internally.

Thank you so much @francispeabody .

---

ALB

15:26:19 ➜ pt-1248 git:(master!) http --follow --verbose OPTIONS $ENDPOINT
OPTIONS /hello HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 0
Host: pt-12-myloa-nmcfguurzny6-1668993808.eu-west-1.elb.amazonaws.com
User-Agent: HTTPie/2.4.0

# Response

HTTP/1.1 204 No Content
Access-Control-Allow-Headers: Authorization,Content-Type,X-Amz-Date,X-Amz-Security-Token,X-Api-Key
Access-Control-Allow-Methods: GET,OPTIONS
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 1
Connection: keep-alive
Date: Thu, 16 Jun 2022 13:26:22 GMT
Server: awselb/2.0

API Gateway

15:26:23 ➜ pt-1248 git:(master!) ENDPOINT="https://420cn1i5v5.execute-api.eu-west-1.amazonaws.com/Prod/hello"

15:26:39 ➜ pt-1248 git:(master!) http --follow --verbose OPTIONS $ENDPOINT
OPTIONS /Prod/hello HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 0
Host: 420cn1i5v5.execute-api.eu-west-1.amazonaws.com
User-Agent: HTTPie/2.4.0


HTTP/1.1 200 OK
Access-Control-Allow-Headers: Content-Type, X-Forwarded-For
Access-Control-Allow-Methods: GET
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 1
Connection: keep-alive
Content-Length: 3
Content-Type: application/json
Date: Thu, 16 Jun 2022 13:26:42 GMT
Via: 1.1 83bc0649a33d85c1cf516bf48779a390.cloudfront.net (CloudFront)
X-Amz-Cf-Id: UExltMRxrJ75RmUg227tCf94qgpMEIGpESVKOewjmsOjt0EaS7cXOQ==
X-Amz-Cf-Pop: AMS1-C1
X-Cache: Miss from cloudfront
x-amz-apigw-id: T0Re4HyqjoEFcDg=
x-amzn-RequestId: 5defa35b-1fe9-4a82-b808-e09a71c248c5

{}

[github-actions]

This is now released under 1.26.2 version!

[heitorlessa]

hey @francispeabody the fix is now out within 1.26.2 version in PyPi/SAR. I've started the release for Lambda Layer version 21 - might take a few hours to hit all commercial regions we support.

[heitorlessa]

Lambda Layer version 21 is now available on the regions we support too:

• arn:aws:lambda:{region}:017000801446:layer:AWSLambdaPowertoolsPython:21

Please feel free to reopen if this is still occurring in the latest version @francispeabody and thank you again for spotting this!

[francispeabody]

@heitorlessa wow you guys are quick. Thanks much. Just tested on my end and confirmed the fix is working. Best of luck