Use a custom sitecustomize.py / wrapper entry point to manipulate sys.path

[brandjon]

There are a few issues that relate to the initialization of sys.path.

• Bazel is using python user site packages by default #4939: User site packages should not be added (e.g., /home/brandjon/.local/lib/python3.6/site-packages)
• Bazel should put external repos after stdlib on PYTHONPATH #5899 / Give imported packages lower precedence than standard library. #6532: Imported packages should have lower precedence than standard library
• py_binary rules default to importing non-sandboxed code #7091: The main module's directory (which is usually in the source root) should not be added to sys.path

We may be able to address some of this by generating a custom sitecustomize.py file, and ensuring it's given priority by manipulating the PYTHONPATH in the stub script.

Another alternative is to insert another level of indirection before the user's entry point, by creating a wrapper module that runs in the same Python interpreter process as the user code, and that initializes sys.path appropriately. This may affect the sys environment as seen by user code, e.g. sys.argv[0], though maybe the wrapper module could cover some of that up.

Then the question is, if we have an in-process wrapper module anyway, can we get rid of the stub script (which runs in a separate Python process) and thereby solve issues like #544? Or is there some initialization work that has to be done before the real Python process is launched, and that would be tedious to offload into a non-Python launcher script?

[bsarden]

Is there a roadmap for this feature? The import package precedence issues seems to cause a lot of non-trivial issues when using python3 for me. (Even when using the new python-toolchain flow).

Update after 27.0 release:
This issue still exists and can be replicated with the following simple test using the absl-py module and bazel_rules_python at commit:

## Rules python definition to show version that is being used
git_repository(
        name = "io_bazel_rules_python",
        remote = "https://github.com/bazelbuild/rules_python.git",
        commit = "fdbb17a4118a1728d19e638a5291b4c4266ea5b8",
)

## Bazel target
py_test(
    name = "py_version_test",
    size = "small",
    srcs = ["py_version_test.py"],
    python_version = "PY3",
    deps = [
        requirement("absl-py"),
    ],
)

## Test file
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import os
import sys
import enum

# print(sys.path)
# print(enum.__file__)
# print(os.environ["PYTHONPATH"])

from absl import app
from absl import logging

def main(argv):
    if sys.version_info[0] < 3:
        print("not using python3")
        # raise Exception("Must be using Python 3")
    logging.info(sys.version_info)

if __name__ == "__main__":
    app.run(main)

[aiuto]

@brandjon Do you think this is still a P2? If so, do you think we should turn it over to ask for help from the community?

[sgowroji]

Hi there! We're doing a clean up of old issues and will be closing this one. Please reopen if you’d like to discuss anything further. We’ll respond as soon as we have the bandwidth/resources to do so.

[matts1]

I think this is even more important now that bzlmod has been released.

If you create a repo @foo, then it generates the directory _main~bar~foo, and an entry in the _repo_mapping file.

At the moment, to the best of my knowledge, there's no way to write from foo import baz, and we'd need a custom entry is sys.meta_path which is runfiles-aware. @Wyverald do you know if there's an existing way to do this? If not could we reopen this bug?

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.