Include complete platforms for FaaS environments for more reliable building

[huonw]

This changes the behaviour of the runtime field of the python_aws_lambda_function, python_aws_lambda_layer and python_google_cloud_function FaaS targets to be translated into a Pants-provided complete platform (if known), rather than using the less specific --platform argument.

For example, when building a target like python_aws_lambda_function(..., runtime="python3.10"), Pants will pull out an appropriate complete platform resource and use that when building the Lambda function, to choose the correct wheels.

This is motivated by:

1. the FaaS runtimes generally being stable (AIUI), so it's actually reasonable/possible to provide this.
2. the naive wheel selection for runtime/--platform often fails and users have to switch to using complete platforms themselves, and, one can still use complete_platforms manually, so this is not worse.
3. the switch to layout = "zip" means that passing both runtime="..." and complete_platforms=[...] will fail, as only one --platform=... or --complete-platform=... argument can be passed to the underlying PEX call.

The moving parts are:

1. a script build-support/bin/generate_faas_complete_platforms.py that reads information about each FaaS system, including runtime docker repo and the tags for each 'known runtime', and shells out to docker to extra a complete platform JSON file
2. if complete_platforms isn't specified, convert the runtime a complete platform argument (if it's known), or continue falling back to the naive --platform style

Notes:

• Notable failing: this is only implemented for AWS: I cannot find an equivalent docker repo to https://gallery.ecr.aws/lambda/python for GCF, so despite being generic infrastructure, there's no 'known' runtimes for GCF.
  • Does anyone know of docker images that simulate the runtime environment of GCF?
  • (I've found https://github.com/GoogleCloudPlatform/python-runtime but it's for AppEngine and is quite old, and https://cloud.google.com/functions/docs/concepts/python-runtime is very cagey about Python patch versions, let alone GLibc/manylinux tags!)
• This changes the interaction between the runtime and complete_platforms field:
  • passing both is deprecated, and the behaviour changes too (runtime is now totally ignored, but, per point 3 of the motivation above, this would fail anyway)
  • passing complete_platforms=[] is now permitted, and, I hope, means the PEX calls will use the ambient interpreter, e.g. if running in a compatible docker image
• I haven't written explicit tests for the inference behaviour yet, as I didn't want to go too deep down that path before validating that this is sensible
• I've only implemented this for layout = "zip", as layout = "lambdex" will be removed soon.
• (Once we have this, I think we can even do one step better: infer an appropriate runtime from interpreter constraints (e.g. if a repo has constraints that cover only one major version, like ==3.10.*, infer runtime="python3.10"), so that a minimal FaaS target can be python_aws_lambda_function(name="foo", handler="./foo.py:handler").)

Fixes #18195

[benjyw]

Excellent!

I don't know how to find out the exact GCF runtime short of actually running a live function to do so?

[huonw]

I'm going to merge this as is, with Lambda support, but keep #18195 open to cover adding it for GCF too.