New env() BUILD file function.

[kaos]

Closes #12797

Make BUILD file parsing a two pass operation, where the first pass is just walking the AST to fish out any env(...) calls so we can provide the requested env vars for the second pass.

Example:

# src/python/some/BUILD
python_distribution(
  name="my-dist",
  description=env("DESC_TEXT", "demo the `env()` function, with default value."),
  provides=python_artifact(
    name="my-dist",
    version=env("MY_DIST_VERSION"),
  )
)

[kaos]

I think it may be good enough to only be able to use the env vars of the local system, even when using remote execution--as long as it's a known and documented limitation.

At least I've not been able to figure out how to make it work otherwise.. ;)

[stuhood]

Great idea!

[kaos]

@benjyw 🎉 your aws s3 fix seems to have helped :)

[thejcannon]

Sorry to snipe this review after merge. Can we document whether sensitive env vars are safe to load? Why/why not?

[kaos]

Sounds like a good idea. I don't know the answer though.. :)

[thejcannon]

@stuhood ?

[stuhood]

I mean... it just depends where they are being used, I suppose? Technically you could use this to interpolate an env var into a target name or something, at which point your secret would leak via ./pants list.

So it's very use-site specific.

[TalAmuyal]

Joining the snipe here, can you document whether or not the env-var value is part of the cache key?
I'd like to use it for the package-version, since it is computed (in our case) based on the build-number.

In this case, in each new build (even for the same commit), the version would change and I wouldn't want it to invalidate the package.

[kaos]

Joining the snipe here, can you document whether or not the env-var value is part of the cache key? I'd like to use it for the package-version, since it is computed (in our case) based on the build-number.

In this case, in each new build (even for the same commit), the version would change and I wouldn't want it to invalidate the package.

When packaging a target, the hash value of that target will be part of the cache key, and if any of the target's fields values change the hash value change.

In other words, yes the env-var value will be part of the cache key--as will it be if injected by any other means using the SetupKwargs plugin API for instance.

Your use case is a bit tricky, as you change some input which has a noticeable effect on the output but wish it to not trigger a rebuild. For that to work, we'd need to have a way to mark values with a constant hash regardless of its underlying value in order to not invalidate previous work when changed.