Always provide Python-for-Pants-scripts (Cherry-pick of #18433)

[thejcannon]

This PR decouples the Python Pants uses for its own nefarious purposes (like running PEX or gunzip) from the user search paths by either using sys.executable locally or downloading and using Python Build Standalone in a Docker environment.

Additionally when making this change, the Python/pex code was refactored so that we always use this Python to run pex, with Python either being chosen by pex, or by using PEX_PYTHON env var at runtime. I think this is a nice cleanup of the handshake between CompletePexEnvironment.create_argv and CompletePexEnvironment.environment_dict used to have.

On a side note, this (with scie-pants and #18352) this marks a complete divorce from any Python on the user system (or lack thereof) and running Python code.

[kaos]

do we have any guiding criteria for when we pick non-bugfixes/doc-changes to release branches?

[thejcannon]

Maybe I'm stretching it a bit, but I meant for this to be in 2.16.x. I just missed the train 😭 .

I'm OK punting though, I just don't want to 😛

[kaos]

I'm fine with it, just want us to be intentional with if this will be an exception or not. (holding out my approve a little to give room for more opinions..)

[stuhood]

I'm fine cherry-picking this as long as we do another 2.16.0a release. But we need to continue to watch out for large picks, and not do them at all after an rc is out... the PEX / python_sources that got picked in 2.15.x were rocky.

[kaos]

I'm fine cherry-picking this as long as we do another 2.16.0a release. But we need to continue to watch out for large picks, and not do them at all after an rc is out... the PEX / python_sources that got picked in 2.15.x were rocky.

Thanks, so that makes sense. No feat picks into rc releases--which then implies that if you pick onto an alpha that will add another alpha before we go to rc. Is there room for this some where in the release strategy docs perhaps?

[stuhood]

I'm fine cherry-picking this as long as we do another 2.16.0a release. But we need to continue to watch out for large picks, and not do them at all after an rc is out... the PEX / python_sources that got picked in 2.15.x were rocky.

Thanks, so that makes sense. No feat picks into rc releases--which then implies that if you pick onto an alpha that will add another alpha before we go to rc. Is there room for this some where in the release strategy docs perhaps?

It is mentioned, and calls for "extenuating circumstances": https://www.pantsbuild.org/docs/release-strategy#alpha-releases

[thejcannon]

Appreciate the understanding y'all. I definitely owe everyone a blog post about this

[kaos]

I'm fine cherry-picking this as long as we do another 2.16.0a release. But we need to continue to watch out for large picks, and not do them at all after an rc is out... the PEX / python_sources that got picked in 2.15.x were rocky.

Thanks, so that makes sense. No feat picks into rc releases--which then implies that if you pick onto an alpha that will add another alpha before we go to rc. Is there room for this some where in the release strategy docs perhaps?

Is it mentioned, and calls for "extenuating circumstances": https://www.pantsbuild.org/docs/release-strategy#alpha-releases

Ah.. coming from this side, that is clear. Having read only that does not make this side clear--so a few extra words, as an example for what an extenuating circumstance might be and what that implies, could be helpful. I'm up for writing it if that sounds sensible.

[stuhood]

Ah.. coming from this side, that is clear. Having read only that does not make this side clear--so a few extra words, as an example for what an extenuating circumstance might be and what that implies, could be helpful. I'm up for writing it if that sounds sensible.

Yea, feel free!