Forward new provider for python in filter_layer

[aaliddell]

In upcoming Bazel 0.23.0, the legacy py provider has been changed to PyInfo, with both being allowed to work currently. However, in a future release the use of py will be not be permitted, which can be simulated with --incompatible_disallow_legacy_py_provider.

This change forwards the old py provider in filter_layer in older Bazel, and PyInfo in newer Bazel and has been tested to work correctly when --incompatible_disallow_legacy_py_provider is enabled. Prior to this change, filter_layer would fail when run in this mode.

[k8s-ci-robot]

Hi @aaliddell. Thanks for your PR.

I'm waiting for a bazelbuild member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[aaliddell]

Ah, unfortunately Starlark behaves slightly differently from what I expected from full Python when using identifiers that are not defined in a not-taken branch. I can't see an obvious way around this to allow picking the correct provider when PyInfo is not defined in Bazel < 0.23.0 and for the time being the py provider is allowed.

Perhaps instead of proxying an externally created py_library in filter_layer, the rule should instead create and return a native.py_library internally after doing the deps filtering, which would itself have the correct python provider, rather than trying to forward it?

[xingao267]

Hi @aaliddell, thanks for sending the PR. We can wait until Bazel 0.23.0 is out before making this change. And once the --incompatible_disallow_legacy_py_provider flag is flipped in either Bazel 0.24.0 or 0.25.0, we can remove forwarding legacy py provider at all.

[aaliddell]

Ok. If you're going to require Bazel 0.23.0 anyway, you might as well always forward the PyInfo provider to match the behaviour of the built-in python rules, such as (not tested):

    # Forward correct provider, depending on availability, so that the filter_layer() can be
    # used as a normal dependency to native targets (e.g. py_library(deps = [<filter_layer>])).
    if hasattr(ctx.attr.dep, "py"):
        # Forward legacy builtin provider and PyInfo provider
        return struct(
            providers = [
                FilterLayerInfo(
                    runfiles = runfiles,
                    filtered_depset = depset(transitive = filtered_depsets),
                ),
            ] + ([ctx.attr.dep[PyInfo]] if PyInfo in ctx.attr.dep else []),
            py = ctx.attr.dep.py if hasattr(ctx.attr.dep, "py") else None,
        )
    else:
        # Forward the PyInfo provider only
        return struct(
            providers = [
                FilterLayerInfo(
                    runfiles = runfiles,
                    filtered_depset = depset(transitive = filtered_depsets),
                ),
            ] + ([ctx.attr.dep[PyInfo]] if PyInfo in ctx.attr.dep else []),
)

Then in 0.24.0 or 0.25.0, the branch containing py can be dropped.

[xingao267]

Yes. We can hold on this until Bazel 0.23.0 is out.

[aaliddell]

I've implemented the proposed change assuming 0.23.0+, which was just released. Once things have settled and the CI version has been updated, this can be retested.

[xingao267]

CI has switched to 0.23.1 and the tests passed. Can you fix the buildifier issue?

[aaliddell]

The buildifier issue appears to have come in with the merge from master and the output on buildkite contains no useful info. Any ideas what it might be complaining about?

[xingao267]

It's because Buildifier on Buildkite CI is upgraded today bazelbuild/continuous-integration@6d48796. Hold on a little on this PR and we will fix it.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: aaliddell, xingao267
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:

If they are not already assigned, you can assign the PR to them by writing /assign in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment