Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an environment= field to pex_binary, and support for environments to the package goal #17106

Closed

Conversation

stuhood
Copy link
Member

@stuhood stuhood commented Oct 4, 2022

pex_binary targets gain an environment= field, which drives the environment in which they are packaged.

This was validated working using the instructions from #17096.

[ci skip-rust]
[ci skip-build-wheels]

…al, and add to `pex_binary`.

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@@ -13,4 +13,5 @@ python_test_utils(name="test_utils")
_docker_environment(
name="docker_env",
image="python:3.9",
python_bootstrap_search_path=["<PATH>"],
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to support the case where [python-bootstrap].search_path is being overridden by a Pants developer to point at <PYENV> instead: this overrides it back to something which is valid for this image.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stuhood Note that for python bootstrapping, keeping the default value of ["<PYENV>", "<PATH>"] is acceptable, the <PYENV> will be discarded, leaving only <PATH>.

<PYENV> in a manually-specified value is not acceptable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<PYENV> in a manually-specified value is not acceptable.

It is what I use by default via a .pants.rc, in order to avoid ever using a system interpreter.

Copy link
Contributor

@chrisjrn chrisjrn Oct 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack.

Obviously in favour of making the _docker_environment config explicitly valid here. I'm just suggesting highlighting this as not strictly necessary given the config in Pants' pants.toml, so it's clear to readers why it's necessary.

@stuhood stuhood changed the title Add an environment= field to pex_binary, and support for environments in the package goal Add an environment= field to pex_binary, and support for environments to the package goal Oct 4, 2022
Copy link
Contributor

@chrisjrn chrisjrn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine. The MultiGet pattern for evaluating against each environment could be more concisely defined, since it'll probably be important for plugin authors too.

@@ -289,9 +290,19 @@ async def create_docker_build_context(request: DockerBuildContextRequest) -> Doc
)

# Package binary dependencies for build context.
embedded_pkgs = await MultiGet(
Get(BuiltPackage, PackageFieldSet, field_set)
environment_names_per_field_set = await MultiGet(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noting that this is happening over in package.py too, it looks like this pattern of requesting a set of objects corresponding to different environments is likely to be a frequent one -- and one that took me a bit of reading to wrap my head around.

Any chance of factoring that out into something reusable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, that's probably worth doing. The three use-cases of environments-on-roots so far all use FieldSetsPerTargetRequest ... will see if it can be applied there generically.

Copy link
Contributor

@Eric-Arellano Eric-Arellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you grep for (BuiltPackage, I think other places need this, like test.py for runtime_package_dependencies, and publish.py

@stuhood
Copy link
Member Author

stuhood commented Oct 5, 2022

To make this less error prone, I'm going to focus on #17129 for now. Moving this back to draft.

@stuhood stuhood marked this pull request as draft October 5, 2022 22:11
@stuhood
Copy link
Member Author

stuhood commented Oct 10, 2022

@chrisjrn is going to tackle this as part of #17155. Closing for now, and he'll reopen with a similar shape to #17186.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants