From 9cbd24c5fbabb2a2a772fb007629bb9dfc617b5c Mon Sep 17 00:00:00 2001 From: Jeroen Schmidt <25979801+JeroenSchmidt@users.noreply.github.com> Date: Thu, 16 Jan 2025 12:16:11 +0100 Subject: [PATCH 1/4] Update docs & Examples --- docs/_includes/py_console_script_binary.md | 18 ++- docs/toolchains.md | 65 ++++++++-- .../multi_python_versions/tests/BUILD.bazel | 120 ++++++++++++------ 3 files changed, 150 insertions(+), 53 deletions(-) diff --git a/docs/_includes/py_console_script_binary.md b/docs/_includes/py_console_script_binary.md index 7373c8a7b2..49983bf25d 100644 --- a/docs/_includes/py_console_script_binary.md +++ b/docs/_includes/py_console_script_binary.md @@ -12,7 +12,8 @@ py_console_script_binary( ) ``` -Or for more advanced setups you can also specify extra dependencies and the +#### Speicfying Extra Dependecies +You can also specify extra dependencies and the exact script name you want to call. It is useful for tools like `flake8`, `pylint`, `pytest`, which have plugin discovery methods and discover dependencies from the PyPI packages available in the `PYTHONPATH`. @@ -34,17 +35,26 @@ py_console_script_binary( ) ``` -A specific Python version can be forced by using the generated version-aware -wrappers, e.g. to force Python 3.9: +#### Using a specific Python Version + +A specific Python version can be forced by passing the desired python version, e.g. to force Python 3.9: ```starlark -load("@python_versions//3.9:defs.bzl", "py_console_script_binary") +load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") py_console_script_binary( name = "yamllint", pkg = "@pip//yamllint", + python_version="3.9" ) ``` +#### Using a specific Python Version directly from a Toolchain +:::{note} +Deprecation warning v1.1.0: The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. +i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` + +You should instead specify the desired python version with `python_version` when using `py_console_script_binary`; see above example. +::: Alternatively, the [`py_console_script_binary.binary_rule`] arg can be passed the version-bound `py_binary` symbol, or any other `py_binary`-compatible rule of your choosing: diff --git a/docs/toolchains.md b/docs/toolchains.md index 32f4a541d9..b0b8b76fb3 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -116,9 +116,9 @@ python = use_extension("@rules_python//python/extensions:python.bzl", "python") python.toolchain(python_version = "3.12") # BUILD.bazel -load("@python_versions//3.12:defs.bzl", "py_binary") +load("@rules_python//python:py_binary.bzl", "py_binary") -py_binary(...) +py_binary(..., python_version="3.12") ``` ### Pinning to a Python version @@ -132,21 +132,60 @@ is most useful for two cases: typically in a mono-repo situation. To configure a submodule with the version-aware rules, request the particular -version you need, then use the `@python_versions` repo to use the rules that -force specific versions: +version you need when defining the toolchain: ```starlark +# MODULE.bazel python = use_extension("@rules_python//python/extensions:python.bzl", "python") python.toolchain( python_version = "3.11", ) -use_repo(python, "python_versions") +use_repo(python) +``` + +Then use the `@rules_python` repo in your BUILD file to explicity pin the python version when calling the rule: + +```starlark +# BUILD.bazel +load("@rules_python//python:py_binary.bzl", "py_binary") + +py_binary(..., python_version="3.11") +py_test(..., python_version="3.11") ``` -Then use e.g. `load("@python_versions//3.11:defs.bzl", "py_binary")` to use -the rules that force that particular version. Multiple versions can be specified -and use within a single build. +Multiple versions can be specified and use within a single build. + +```starlark +# MODULE.bazel +python = use_extension("@rules_python//python/extensions:python.bzl", "python") + +python.toolchain( + python_version = "3.11", + is_default = True, +) + +python.toolchain( + python_version = "3.12", +) +use_repo(python) + +# BUILD.bazel +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") + +# Defaults to 3.11 +py_binary(...) +py_test(...) + +# Explicitly using python 3.11 +py_binary(..., python_version="3.11") +py_test(..., python_version="3.11") + +# Explicitly using python 3.12 +py_binary(..., python_version="3.12") +py_test(..., python_version="3.12") +``` For more documentation, see the bzlmod examples under the {gh-path}`examples` folder. Look for the examples that contain a `MODULE.bazel` file. @@ -159,6 +198,16 @@ The `python.toolchain()` call makes its contents available under a repo named Remember to call `use_repo()` to make repos visible to your module: `use_repo(python, "python_3_11")` + +:::{note} +Deprecation warning v1.1.0: The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. +i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` + +Usages of them should be changed to load the regular rules directly; +i.e. Use `load("@rules_python//python:py_binary.bzl", "py_binary")` & `load("@rules_python//python:py_test.bzl", "py_test")` and then specify the `python_version` when using the rules corresponding to the python version you defined in your toolchain. {ref}`Library modules with version constraints` +::: + + #### Toolchain usage in other rules Python toolchains can be utilized in other bazel rules, such as `genrule()`, by diff --git a/examples/multi_python_versions/tests/BUILD.bazel b/examples/multi_python_versions/tests/BUILD.bazel index d04ac6bb0a..4491fe3626 100644 --- a/examples/multi_python_versions/tests/BUILD.bazel +++ b/examples/multi_python_versions/tests/BUILD.bazel @@ -1,10 +1,6 @@ load("@bazel_skylib//rules:copy_file.bzl", "copy_file") load("@bazel_skylib//rules:diff_test.bzl", "diff_test") load("@bazel_skylib//rules:write_file.bzl", "write_file") -load("@python//3.10:defs.bzl", py_binary_3_10 = "py_binary", py_test_3_10 = "py_test") -load("@python//3.11:defs.bzl", py_binary_3_11 = "py_binary", py_test_3_11 = "py_test") -load("@python//3.8:defs.bzl", py_binary_3_8 = "py_binary", py_test_3_8 = "py_test") -load("@python//3.9:defs.bzl", py_binary_3_9 = "py_binary", py_test_3_9 = "py_test") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING", "PYTHON_VERSIONS") load("@rules_python//python:py_binary.bzl", "py_binary") load("@rules_python//python:py_test.bzl", "py_test") @@ -12,79 +8,91 @@ load("@rules_python//python:versions.bzl", DEFAULT_MINOR_MAPPING = "MINOR_MAPPIN load("@rules_python//python/private:text_util.bzl", "render") # buildifier: disable=bzl-visibility load("@rules_shell//shell:sh_test.bzl", "sh_test") -copy_file( - name = "copy_version", - src = "version.py", - out = "version_default.py", - is_executable = True, -) -# NOTE: We are testing that the `main` is an optional param as per official -# docs https://bazel.build/reference/be/python#py_binary.main +################################################################################ +# Defining py_binary with different versions + py_binary( name = "version_default", - srcs = ["version_default.py"], + srcs = ["version.py"], + main = "version.py", + # NOTE: # The default python_version used by this rule is defined in the WORKSPACE / MODULE.bazel ) -py_binary_3_8( +py_binary( name = "version_3_8", srcs = ["version.py"], main = "version.py", + # NOTE: The value specified with python_version must be a toolchain configured to match the Python version + python_version = "3.8" ) -py_binary_3_9( +py_binary( name = "version_3_9", srcs = ["version.py"], main = "version.py", + python_version = "3.9" ) -py_binary_3_10( +py_binary( name = "version_3_10", srcs = ["version.py"], main = "version.py", + python_version = "3.10" ) -py_binary_3_11( +py_binary( name = "version_3_11", srcs = ["version.py"], main = "version.py", + python_version = "3.11" ) +################################################################################ +# Defining py_test with different versions with deps + py_test( name = "my_lib_default_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", deps = ["//libs/my_lib"], + # NOTE: # The default python_version used by this rule is defined in the WORKSPACE / MODULE.bazel ) -py_test_3_8( +py_test( name = "my_lib_3_8_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", deps = ["//libs/my_lib"], + python_version = "3.8" ) -py_test_3_9( +py_test( name = "my_lib_3_9_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", deps = ["//libs/my_lib"], + python_version = "3.9" ) -py_test_3_10( +py_test( name = "my_lib_3_10_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", deps = ["//libs/my_lib"], + python_version = "3.10" ) -py_test_3_11( +py_test( name = "my_lib_3_11_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", deps = ["//libs/my_lib"], ) +################################################################################ +# Sanity check that py_test is using the expected python verion + copy_file( name = "copy_version_test", src = "version_test.py", @@ -95,39 +103,47 @@ copy_file( py_test( name = "version_default_test", srcs = ["version_default_test.py"], - env = {"VERSION_CHECK": "3.9"}, # The default defined in the WORKSPACE. + # The default python_version used by this rule is defined in the WORKSPACE / MODULE.bazel + env = {"VERSION_CHECK": "3.9"}, ) -py_test_3_8( - name = "version_3_8_test", +py_test( + name = "version_3_9_test", srcs = ["version_test.py"], - env = {"VERSION_CHECK": "3.8"}, + env = {"VERSION_CHECK": "3.9"}, main = "version_test.py", + python_version = "3.9" # We can explicity mention the python version instead of using the default python version ) -py_test_3_9( - name = "version_3_9_test", +py_test( + name = "version_3_8_test", srcs = ["version_test.py"], - env = {"VERSION_CHECK": "3.9"}, + env = {"VERSION_CHECK": "3.8"}, main = "version_test.py", + python_version = "3.8" ) -py_test_3_10( +py_test( name = "version_3_10_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.10"}, main = "version_test.py", + python_version = "3.10" ) -py_test_3_11( +py_test( name = "version_3_11_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.11"}, main = "version_test.py", + python_version = "3.11" ) +################################################################################ +# Cross Version Target Usage + py_test( - name = "version_default_takes_3_10_subprocess_test", + name = "python_version_default_takes_3_10_subprocess_test", srcs = ["cross_version_test.py"], data = [":version_3_10"], env = { @@ -138,8 +154,8 @@ py_test( main = "cross_version_test.py", ) -py_test_3_10( - name = "version_3_10_takes_3_9_subprocess_test", +py_test( + name = "python_version_3_10_takes_3_9_subprocess_test", srcs = ["cross_version_test.py"], data = [":version_3_9"], env = { @@ -148,25 +164,47 @@ py_test_3_10( "VERSION_CHECK": "3.10", }, main = "cross_version_test.py", + python_version = "3.10" ) +################################################################################ +# Using Multiple Python Versions with sh_test + sh_test( - name = "version_test_binary_default", + name = "version_test_binary_3_8", srcs = ["version_test.sh"], - data = [":version_default"], + data = [":version_3_8"], env = { - "VERSION_CHECK": "3.9", # The default defined in the WORKSPACE. - "VERSION_PY_BINARY": "$(rootpaths :version_default)", + # The python version is transiently obtained from the py_binary target :version_3_8; which in this case is 3.8 + "VERSION_CHECK": "3.8", + "VERSION_PY_BINARY": "$(rootpaths :version_3_8)", }, ) +copy_file( + name = "copy_version", + src = "version.py", + out = "version_default.py", + is_executable = True, +) + +# NOTE: We are testing that the `main` is an optional param as per official +# docs https://bazel.build/reference/be/python#py_binary.main +py_binary( + name = "version_default", + srcs = ["version_default.py"], + # The default python_version used by this rule is defined in the WORKSPACE / MODULE.bazel +) + sh_test( - name = "version_test_binary_3_8", + name = "version_test_binary_default", srcs = ["version_test.sh"], - data = [":version_3_8"], + data = [":version_default"], env = { - "VERSION_CHECK": "3.8", - "VERSION_PY_BINARY": "$(rootpaths :version_3_8)", + # The python version is transiently obtained from the py_binary target :version_default, + # which happens to be the default python_version defined by the toolchain in workspace/MODULE.bazel + "VERSION_CHECK": "3.9", + "VERSION_PY_BINARY": "$(rootpaths :version_default)", }, ) From 0194a6120604fb297cb682462d140533db80e86c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 16 Jan 2025 16:32:03 -0800 Subject: [PATCH 2/4] Apply suggestions from code review --- docs/_includes/py_console_script_binary.md | 14 +++++++------- docs/toolchains.md | 15 +++++++-------- 2 files changed, 14 insertions(+), 15 deletions(-) diff --git a/docs/_includes/py_console_script_binary.md b/docs/_includes/py_console_script_binary.md index 49983bf25d..5d72f067ee 100644 --- a/docs/_includes/py_console_script_binary.md +++ b/docs/_includes/py_console_script_binary.md @@ -12,7 +12,7 @@ py_console_script_binary( ) ``` -#### Speicfying Extra Dependecies +#### Specifying extra dependencies You can also specify extra dependencies and the exact script name you want to call. It is useful for tools like `flake8`, `pylint`, `pytest`, which have plugin discovery methods and discover dependencies from the @@ -35,7 +35,7 @@ py_console_script_binary( ) ``` -#### Using a specific Python Version +#### Using a specific Python version A specific Python version can be forced by passing the desired python version, e.g. to force Python 3.9: ```starlark @@ -44,16 +44,16 @@ load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_cons py_console_script_binary( name = "yamllint", pkg = "@pip//yamllint", - python_version="3.9" + python_version = "3.9" ) ``` #### Using a specific Python Version directly from a Toolchain -:::{note} -Deprecation warning v1.1.0: The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. -i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` +:::{deprecated} 1.1.0 +The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. +i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` and `load("@python_versions//3.11:defs.bzl", "py_test")` -You should instead specify the desired python version with `python_version` when using `py_console_script_binary`; see above example. +You should instead specify the desired python version with `python_version`; see above example. ::: Alternatively, the [`py_console_script_binary.binary_rule`] arg can be passed the version-bound `py_binary` symbol, or any other `py_binary`-compatible rule diff --git a/docs/toolchains.md b/docs/toolchains.md index b0b8b76fb3..b14fb16b3f 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -144,17 +144,17 @@ python.toolchain( use_repo(python) ``` -Then use the `@rules_python` repo in your BUILD file to explicity pin the python version when calling the rule: +Then use the `@rules_python` repo in your BUILD file to explicity pin the Python version when calling the rule: ```starlark # BUILD.bazel load("@rules_python//python:py_binary.bzl", "py_binary") -py_binary(..., python_version="3.11") -py_test(..., python_version="3.11") +py_binary(..., python_version = "3.11") +py_test(..., python_version = "3.11") ``` -Multiple versions can be specified and use within a single build. +Multiple versions can be specified and used within a single build. ```starlark # MODULE.bazel @@ -168,7 +168,6 @@ python.toolchain( python.toolchain( python_version = "3.12", ) -use_repo(python) # BUILD.bazel load("@rules_python//python:py_binary.bzl", "py_binary") @@ -178,7 +177,7 @@ load("@rules_python//python:py_test.bzl", "py_test") py_binary(...) py_test(...) -# Explicitly using python 3.11 +# Explicitly use Python 3.11 py_binary(..., python_version="3.11") py_test(..., python_version="3.11") @@ -199,8 +198,8 @@ Remember to call `use_repo()` to make repos visible to your module: `use_repo(python, "python_3_11")` -:::{note} -Deprecation warning v1.1.0: The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. +:::{deprecated} 1.1.0 +The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` Usages of them should be changed to load the regular rules directly; From e9354c76a6986ef9e4a3ceb31c9e12062a407e7a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 16 Jan 2025 16:32:59 -0800 Subject: [PATCH 3/4] Apply suggestions from code review --- docs/_includes/py_console_script_binary.md | 2 +- docs/toolchains.md | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/_includes/py_console_script_binary.md b/docs/_includes/py_console_script_binary.md index 5d72f067ee..73aa282498 100644 --- a/docs/_includes/py_console_script_binary.md +++ b/docs/_includes/py_console_script_binary.md @@ -37,7 +37,7 @@ py_console_script_binary( #### Using a specific Python version -A specific Python version can be forced by passing the desired python version, e.g. to force Python 3.9: +A specific Python version can be forced by passing the desired Python version, e.g. to force Python 3.9: ```starlark load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") diff --git a/docs/toolchains.md b/docs/toolchains.md index b14fb16b3f..8b0303d5e0 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -178,12 +178,12 @@ py_binary(...) py_test(...) # Explicitly use Python 3.11 -py_binary(..., python_version="3.11") -py_test(..., python_version="3.11") +py_binary(..., python_version = "3.11") +py_test(..., python_version = "3.11") # Explicitly using python 3.12 -py_binary(..., python_version="3.12") -py_test(..., python_version="3.12") +py_binary(..., python_version = "3.12") +py_test(..., python_version = "3.12") ``` For more documentation, see the bzlmod examples under the {gh-path}`examples` From e5624bd5aafe288433913974a97555dec0b35246 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 16 Jan 2025 16:33:10 -0800 Subject: [PATCH 4/4] Apply suggestions from code review --- docs/toolchains.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/toolchains.md b/docs/toolchains.md index 8b0303d5e0..7aa9acc4a1 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -181,7 +181,7 @@ py_test(...) py_binary(..., python_version = "3.11") py_test(..., python_version = "3.11") -# Explicitly using python 3.12 +# Explicitly use Python 3.12 py_binary(..., python_version = "3.12") py_test(..., python_version = "3.12") ```