Implement API keys to control usage

[nikki-t]

Working from the initial architectural review found in #152 we have decided to implement:

1. Enable API keys for the Hydrocron API and create two keys, one for default users and another for more specialized or heavy use.
2. Create usage plans for each API key starting with expected access patterns and cost modelling for defaults.
3. Create a Lambda authorizer which will facilitate the use of API keys and usage plans to throttle requests and set quotas.

[nikki-t]

We need to decide on throttling and quotas.

For throttling we need to decide the Rate (requests per second, that clients can call your API) and Burst (number of concurrent requests that a client can make to your API). We can ask our collaborators for feedback on their expected usage patterns during our meeting on 6/12.

For quotas, I took a look at how many requests would cost ~$5,000 per month and it looks like that is equivalent to 1.8 billion requests per month. Pricing calculator estimates:

1.80 requests x 1,000,000,000 unit multiplier = 1,800,000,000 total REST API requests
Tiered price for: 1,800,000,000 requests
333,000,000 requests x 0.0000035 USD = 1,165.50 USD
667,000,000 requests x 0.0000028 USD = 1,867.60 USD
800,000,000 requests x 0.00000238 USD = 1,904.00 USD
Total tier cost: 1,165.50 USD + 1,867.60 USD + 1,904.00 USD = 4,937.10 USD (REST API requests)
Tiered price total for REST API requests: 4,937.10 USD
0 USD per hour x 730 hours in a month = 0.00 USD for cache memory
Dedicated cache memory total price: 0.00 USD
REST API cost (monthly): 4,937.10 USD

What I am less sure on is the cost for data transfer which seems to be indicated in the examples at this pricing documentation but there is no place to enter expected data transfer sizes in the pricing calculator. So I am not sure how this might be charged.

So I propose starting with 1.8 billion requests in our default usage plan and then splitting up the requests or adding an additional number of requests to the heavy users usage plan. Then we can track costs and adjust as needed. @frankinspace - Is this in line with what you were thinking? Should we set our quota a bit lower to allow for data transfer charges?

[frankinspace]

@nikki-t that makes sense to me, maybe round it off to 2 billion req/month as the cap. Although, do we need to take into account the dynamo read access? Each request runs at least 1 dynamo query so that may need to be factored in as well.

Also would be interested in @jimmie perspective on the costing front. What we're basically saying is that as a project, PO.DAAC is willing to spend up to ~$5k per month to operate the hydrocron API. I don't know if we have any guidance on what that cap should necessarily be from the ESDS program

[nikki-t]

@frankinspace - I created a AWS Pricing Calculator estimate that includes 2 billion API requests and 1 billion DynamoDB reach reads and 1 billion DynamoDB node reads. I based this off our tables in the OPS environment so we might have some realistic item sizes to work with.

I wasn't sure how to quantify reach versus node requests so I selected 1 billion reads for both as our total requests quota will start at 2 billion so 1 billion reach requests and 1 billion node requests. Note, this does not include table storage.

Total monthly cost is: $5,913.10 USD.

I think it's okay to start off with 2 billion requests and then refine based on actual monthly costs but I will hold off on defining anything until @jimmie ways in on costs.

[frankinspace]

Will start with default plan limits:

200 concurrent requests
2,000 requests per second
2,000,000,000 requests per month

[nikki-t]

To confirm our choices I took a look at the API Gateway Dashboard metrics.

Here is an overview of our metrics since April 2024.

Looking further at the number or requests or API calls. I narrowed into the month of June and saw ~750 API calls per minute each day at ~08:00 UTC which is the max we see (we start to see this trend around May 29th with a max of ~1100 requests per minute). I think that works out to ~12.5 calls per second so we should be fine to implement the above limits assuming I am interpreting the metrics correctly.

[frankinspace]

Thanks @nikki-t; I agree those metrics appear to support our plan

[frankinspace]

Implemented in https://github.com/podaac/hydrocron/releases/tag/1.3.0