Add API keys to help control usage

[frankinspace]

API Gateway has a built in mechanism for throttling/limiting individuals called usage plans:
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-api-usage-plans.html

We may need to consider configuring this for hydrocron as currently, there are no usage limits other than the AWS imposed account-level limits (10,000 requests per second (RPS) with an additional burst capacity provided by the token bucket algorithm, using a maximum bucket capacity of 5,000 requests.)

We don't necessarily need to require 1 API key per user or to tie them to an EDL account. But this would be the quickest way to help throttle requests if we see API usage spikes.

[nikki-t]

I did a little bit of a deep dive into API keys and usage plans and came up with three possible solutions. I would like to test whichever we prefer before making any final decisions as I think there is a bit of complexity to this.

1) API keys & usage plan

I think we can definitely set up API keys and hand them out to folks or hand out a single API that we use to limit requests. They would need to make sure the API key was present in the x-api-key header. This presents a little bit of a hurdle for users who will need to obtain and use the API key.

We can also use a Lambda authorizer to limit requests with a usage plan which would still need to use an API key but I am thinking we could use a single API key referenced in the Lambda and therefore the user would not have to obtain and use an API key. The Lambda authorizer must return the API key as a part of its output. You configure a Lambda authorizer for an API gateway method.

You can set rate (requests per second) and burst (I am not sure what this setting does) for the usage plan as well as a quota which is the total number of requests a user can make in the time of period of the plan.

Pros

• Minimal impact to the user, internal user of API keys.
• Set us up to use EDL tokens if we need to.
• I believe that the only additional cost would be the cost to run the Lambda authorizer.

Cons

• I do wonder how this might impact response times though.
• I think this has to be set up for each method but I bet each method can point to the same authorizer.

2) Setting up stage-level throttling

We can define a rate in requests per second and a target burst rate which is the capacity of the token bucket. This operates on the concept of a token bucket algorithm, where a token counts for a request.

Pros

• Simple to implement.
• We do not have to hand out and manage API keys.

Cons

• Does not provide as much flexibility for API usage.

For 1 and 2 - I think the user can receive a 429 Too Many Requests error response which they can then handle and try to submit requests in a way that does not bump up against the limit. I think this makes either option more user friendly. See documentation on throttling API requests.

3) Web Application Firewall (WAF)

You protect resources like the API gateway with web access controls lists (ACL) where each ACL has a rule or rules associated with them. These rules tell the AWS WAF how to inspect the web request. You can then make decisions based on this inspection.

You can aggregate requests by their IP address and any other information that gets passed in the request and rate limit those criteria.

Pros

• We do not have to hand out and manage API keys.
• Less likely to impact performance than option 1, maybe?

Cons

• If we rate limit by the forwarded IP address the header can be modified along the way circumventing the rate limit.
• Additional costs.
  

@frankinspace - What do you think might be best to try and test out first?

[frankinspace]

Going with the Lambda authorizer approach to start.

Will have 2 keys to start with: default/general public and "trusted" partners.

@nikki-t is working on prototyping in SIT

Current problem is that the API key from end users does not get passed through to hydrocron API; it gets block by the platform api gateway at the moment. Need an NGAP ticket to discuss passing API key to tenant API gateway.

[nikki-t]

We have decided on an architecture and will proceed with implementation in #186 and #187.