Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: jvolkman/rules_pycross
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.5.4
Choose a base ref
...
head repository: jvolkman/rules_pycross
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v0.6.0
Choose a head ref
  • 4 commits
  • 39 files changed
  • 2 contributors

Commits on Apr 29, 2024

  1. Drop macos + amd64

    jvolkman committed Apr 29, 2024
    Copy the full SHA
    7547600 View commit details

Commits on May 16, 2024

  1. feat: expose package_annotations (#96)

    * feat: expose package_annotations
    
    * feat: exclude files during wheel installation
    betaboon authored May 16, 2024
    Copy the full SHA
    5ce7a49 View commit details

Commits on May 28, 2024

  1. refactor(bzlmod): mark lock_import and lock_repos as reproducible (#99)

    betaboon authored May 28, 2024
    Copy the full SHA
    531c8d2 View commit details

Commits on Jun 12, 2024

  1. Bump repairwheel to 0.3.1 and pass the new --no-sys-paths param (#102)

    * Bump repairwheel to 0.3.0 and pass the new --no-sys-paths param
    
    * Use 0.3.1
    jvolkman authored Jun 12, 2024
    Copy the full SHA
    70056be View commit details
Showing with 1,673 additions and 375 deletions.
  1. +0 −2 .bcr/presubmit.yml
  2. +9 −0 CHANGELOG.md
  3. +1 −1 MODULE.bazel
  4. +3 −1 docs/ext_lock_import.md
  5. +34 −9 docs/rules.md
  6. +33 −32 e2e/workspace/WORKSPACE
  7. +17 −16 e2e/workspace/testlib.bzl
  8. +1 −0 examples/bzlmod/BUILD.bazel
  9. +9 −0 examples/bzlmod/MODULE.bazel
  10. +299 −2 examples/bzlmod/pdm.lock
  11. +1 −0 examples/bzlmod/pyproject.toml
  12. +29 −24 examples/external_linking/BUILD.bazel
  13. +2 −0 examples/lock_repo/BUILD.bazel
  14. +15 −0 examples/lock_repo/WORKSPACE
  15. +366 −1 examples/lock_repo/pdm.lock
  16. +4 −0 examples/lock_repo/pyproject.toml
  17. +86 −58 examples/pdm/BUILD.bazel
  18. +272 −0 examples/pdm/example_lock.bzl
  19. +208 −1 examples/pdm/pdm.lock
  20. +1 −0 examples/pdm/pyproject.toml
  21. +1 −0 examples/pdm/tools/BUILD.bazel
  22. +37 −28 examples/poetry/BUILD.bazel
  23. +19 −22 pdm.lock
  24. +2 −0 pycross/defs.bzl
  25. +2 −0 pycross/private/bzlmod/BUILD.bazel
  26. +14 −12 pycross/private/bzlmod/lock_import.bzl
  27. +5 −0 pycross/private/bzlmod/lock_repos.bzl
  28. +3 −0 pycross/private/bzlmod/tag_attrs.bzl
  29. +28 −25 pycross/private/lock_attrs.bzl
  30. +7 −1 pycross/private/lock_file.bzl
  31. +60 −65 pycross/private/pycross_deps.lock.bzl
  32. +7 −0 pycross/private/resolved_lock_repo.bzl
  33. +1 −0 pycross/private/tools/lock_model.py
  34. +38 −73 pycross/private/tools/raw_lock_resolver.py
  35. +1 −0 pycross/private/tools/repairwheel/repair_wheel_hook.py
  36. +7 −0 pycross/private/tools/resolved_lock_renderer.py
  37. +44 −1 pycross/private/tools/wheel_installer.py
  38. +6 −0 pycross/private/wheel_library.bzl
  39. +1 −1 pyproject.toml
2 changes: 0 additions & 2 deletions .bcr/presubmit.yml
Original file line number Diff line number Diff line change
@@ -2,7 +2,6 @@ matrix:
platform:
- debian10
- ubuntu2004
- macos
- macos_arm64
bazel:
- 7.x
@@ -20,7 +19,6 @@ bcr_test_module:
platform:
- debian10
- ubuntu2004
- macos
- macos_arm64
tasks:
run_test_module_bazel_6:
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -6,6 +6,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [Unreleased]

### Added

- Adds `install_exclude_globs` to exclude certain files during installation of wheels.

### Changed

- **BREAKING** Introduce `package_annotation` which replaces `always_build_packages`, `build_target_overrides`,
`package_build_dependencies` and `package_ignore_dependencies`.

## [0.5.3]

### Fixed
2 changes: 1 addition & 1 deletion MODULE.bazel
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@ module(
)

bazel_dep(name = "aspect_bazel_lib", version = "1.38.1")
bazel_dep(name = "bazel_features", version = "1.1.1")
bazel_dep(name = "bazel_features", version = "1.11.0")
bazel_dep(name = "bazel_skylib", version = "1.4.2")
bazel_dep(name = "platforms", version = "0.0.4")
bazel_dep(name = "rules_python", version = "0.29.0")
4 changes: 3 additions & 1 deletion docs/ext_lock_import.md
Original file line number Diff line number Diff line change
@@ -14,7 +14,8 @@ lock_import.import_pdm(<a href="#lock_import.import_pdm-all_development_groups">
<a href="#lock_import.import_pdm-require_static_urls">require_static_urls</a>, <a href="#lock_import.import_pdm-target_environments">target_environments</a>)
lock_import.import_poetry(<a href="#lock_import.import_poetry-default_alias_single_version">default_alias_single_version</a>, <a href="#lock_import.import_poetry-disallow_builds">disallow_builds</a>, <a href="#lock_import.import_poetry-local_wheels">local_wheels</a>, <a href="#lock_import.import_poetry-lock_file">lock_file</a>,
<a href="#lock_import.import_poetry-project_file">project_file</a>, <a href="#lock_import.import_poetry-repo">repo</a>, <a href="#lock_import.import_poetry-target_environments">target_environments</a>)
lock_import.package(<a href="#lock_import.package-name">name</a>, <a href="#lock_import.package-always_build">always_build</a>, <a href="#lock_import.package-build_dependencies">build_dependencies</a>, <a href="#lock_import.package-build_target">build_target</a>, <a href="#lock_import.package-ignore_dependencies">ignore_dependencies</a>, <a href="#lock_import.package-repo">repo</a>)
lock_import.package(<a href="#lock_import.package-name">name</a>, <a href="#lock_import.package-always_build">always_build</a>, <a href="#lock_import.package-build_dependencies">build_dependencies</a>, <a href="#lock_import.package-build_target">build_target</a>, <a href="#lock_import.package-ignore_dependencies">ignore_dependencies</a>,
<a href="#lock_import.package-install_exclude_globs">install_exclude_globs</a>, <a href="#lock_import.package-repo">repo</a>)
</pre>


@@ -77,6 +78,7 @@ Specify package-specific settings.
| <a id="lock_import.package-build_dependencies"></a>build_dependencies | A list of additional package keys (name or name@version) to use when building this package from source. | List of strings | optional | `[]` |
| <a id="lock_import.package-build_target"></a>build_target | An optional override build target to use when and if this package needs to be built from source. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | `None` |
| <a id="lock_import.package-ignore_dependencies"></a>ignore_dependencies | A list of package keys (name or name@version) to drop from this package's set of declared dependencies. | List of strings | optional | `[]` |
| <a id="lock_import.package-install_exclude_globs"></a>install_exclude_globs | A list of globs for files to exclude during installation. | List of strings | optional | `[]` |
| <a id="lock_import.package-repo"></a>repo | The repository name | String | required | |


43 changes: 34 additions & 9 deletions docs/rules.md
Original file line number Diff line number Diff line change
@@ -7,10 +7,9 @@ Public build rule API re-exports
## pycross_lock_file

<pre>
pycross_lock_file(<a href="#pycross_lock_file-name">name</a>, <a href="#pycross_lock_file-out">out</a>, <a href="#pycross_lock_file-always_build_packages">always_build_packages</a>, <a href="#pycross_lock_file-always_include_sdist">always_include_sdist</a>, <a href="#pycross_lock_file-build_target_overrides">build_target_overrides</a>,
<a href="#pycross_lock_file-default_alias_single_version">default_alias_single_version</a>, <a href="#pycross_lock_file-disallow_builds">disallow_builds</a>, <a href="#pycross_lock_file-fully_qualified_environment_labels">fully_qualified_environment_labels</a>,
<a href="#pycross_lock_file-generate_file_map">generate_file_map</a>, <a href="#pycross_lock_file-local_wheels">local_wheels</a>, <a href="#pycross_lock_file-lock_model_file">lock_model_file</a>, <a href="#pycross_lock_file-package_build_dependencies">package_build_dependencies</a>,
<a href="#pycross_lock_file-package_ignore_dependencies">package_ignore_dependencies</a>, <a href="#pycross_lock_file-pypi_index">pypi_index</a>, <a href="#pycross_lock_file-remote_wheels">remote_wheels</a>, <a href="#pycross_lock_file-repo_prefix">repo_prefix</a>,
pycross_lock_file(<a href="#pycross_lock_file-name">name</a>, <a href="#pycross_lock_file-out">out</a>, <a href="#pycross_lock_file-always_include_sdist">always_include_sdist</a>, <a href="#pycross_lock_file-annotations">annotations</a>, <a href="#pycross_lock_file-default_alias_single_version">default_alias_single_version</a>,
<a href="#pycross_lock_file-disallow_builds">disallow_builds</a>, <a href="#pycross_lock_file-fully_qualified_environment_labels">fully_qualified_environment_labels</a>, <a href="#pycross_lock_file-generate_file_map">generate_file_map</a>,
<a href="#pycross_lock_file-local_wheels">local_wheels</a>, <a href="#pycross_lock_file-lock_model_file">lock_model_file</a>, <a href="#pycross_lock_file-pypi_index">pypi_index</a>, <a href="#pycross_lock_file-remote_wheels">remote_wheels</a>, <a href="#pycross_lock_file-repo_prefix">repo_prefix</a>,
<a href="#pycross_lock_file-target_environments">target_environments</a>)
</pre>

@@ -23,17 +22,14 @@ pycross_lock_file(<a href="#pycross_lock_file-name">name</a>, <a href="#pycross_
| :------------- | :------------- | :------------- | :------------- | :------------- |
| <a id="pycross_lock_file-name"></a>name | A unique name for this target. | <a href="https://bazel.build/concepts/labels#target-names">Name</a> | required | |
| <a id="pycross_lock_file-out"></a>out | The output file. | <a href="https://bazel.build/concepts/labels">Label</a> | required | |
| <a id="pycross_lock_file-always_build_packages"></a>always_build_packages | A list of package keys (name or name@version) to always build from source. | List of strings | optional | `[]` |
| <a id="pycross_lock_file-always_include_sdist"></a>always_include_sdist | Always include an entry for a package's sdist if one exists. | Boolean | optional | `False` |
| <a id="pycross_lock_file-build_target_overrides"></a>build_target_overrides | A mapping of package keys (name or name@version) to existing pycross_wheel_build build targets. | <a href="https://bazel.build/rules/lib/dict">Dictionary: String -> String</a> | optional | `{}` |
| <a id="pycross_lock_file-annotations"></a>annotations | Optional annotations to apply to packages. | <a href="https://bazel.build/rules/lib/dict">Dictionary: String -> String</a> | optional | `{}` |
| <a id="pycross_lock_file-default_alias_single_version"></a>default_alias_single_version | Generate aliases for all packages that have a single version in the lock file. | Boolean | optional | `False` |
| <a id="pycross_lock_file-disallow_builds"></a>disallow_builds | Do not allow pycross_wheel_build targets in the final lock file (i.e., require wheels). | Boolean | optional | `False` |
| <a id="pycross_lock_file-fully_qualified_environment_labels"></a>fully_qualified_environment_labels | Generate fully-qualified environment labels. | Boolean | optional | `True` |
| <a id="pycross_lock_file-generate_file_map"></a>generate_file_map | Generate a FILES dict containing a mapping of filenames to repo labels. | Boolean | optional | `False` |
| <a id="pycross_lock_file-local_wheels"></a>local_wheels | A list of wheel files. | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` |
| <a id="pycross_lock_file-lock_model_file"></a>lock_model_file | The lock model JSON file. | <a href="https://bazel.build/concepts/labels">Label</a> | required | |
| <a id="pycross_lock_file-package_build_dependencies"></a>package_build_dependencies | A dict of package keys (name or name@version) to a list of that packages build dependency keys. | <a href="https://bazel.build/rules/lib/dict">Dictionary: String -> List of strings</a> | optional | `{}` |
| <a id="pycross_lock_file-package_ignore_dependencies"></a>package_ignore_dependencies | A dict of package keys (name or name@version) to a list of that packages dependency keys to ignore. | <a href="https://bazel.build/rules/lib/dict">Dictionary: String -> List of strings</a> | optional | `{}` |
| <a id="pycross_lock_file-pypi_index"></a>pypi_index | The PyPI-compatible index to use (must support the JSON API). | String | optional | `""` |
| <a id="pycross_lock_file-remote_wheels"></a>remote_wheels | A mapping of remote wheels to their sha256 hashes. | <a href="https://bazel.build/rules/lib/dict">Dictionary: String -> String</a> | optional | `{}` |
| <a id="pycross_lock_file-repo_prefix"></a>repo_prefix | The prefix to apply to repository targets. Defaults to the lock file target name. | String | optional | `""` |
@@ -151,7 +147,8 @@ pycross_wheel_build(<a href="#pycross_wheel_build-name">name</a>, <a href="#pycr
## pycross_wheel_library

<pre>
pycross_wheel_library(<a href="#pycross_wheel_library-name">name</a>, <a href="#pycross_wheel_library-deps">deps</a>, <a href="#pycross_wheel_library-enable_implicit_namespace_pkgs">enable_implicit_namespace_pkgs</a>, <a href="#pycross_wheel_library-python_version">python_version</a>, <a href="#pycross_wheel_library-wheel">wheel</a>)
pycross_wheel_library(<a href="#pycross_wheel_library-name">name</a>, <a href="#pycross_wheel_library-deps">deps</a>, <a href="#pycross_wheel_library-enable_implicit_namespace_pkgs">enable_implicit_namespace_pkgs</a>, <a href="#pycross_wheel_library-install_exclude_globs">install_exclude_globs</a>,
<a href="#pycross_wheel_library-python_version">python_version</a>, <a href="#pycross_wheel_library-wheel">wheel</a>)
</pre>


@@ -164,6 +161,7 @@ pycross_wheel_library(<a href="#pycross_wheel_library-name">name</a>, <a href="#
| <a id="pycross_wheel_library-name"></a>name | A unique name for this target. | <a href="https://bazel.build/concepts/labels#target-names">Name</a> | required | |
| <a id="pycross_wheel_library-deps"></a>deps | A list of this wheel's Python library dependencies. | <a href="https://bazel.build/concepts/labels">List of labels</a> | optional | `[]` |
| <a id="pycross_wheel_library-enable_implicit_namespace_pkgs"></a>enable_implicit_namespace_pkgs | If true, disables conversion of native namespace packages into pkg-util style namespace packages. When set all py_binary and py_test targets must specify either `legacy_create_init=False` or the global Bazel option `--incompatible_default_to_explicit_init_py` to prevent `__init__.py` being automatically generated in every directory. This option is required to support some packages which cannot handle the conversion to pkg-util style. | Boolean | optional | `True` |
| <a id="pycross_wheel_library-install_exclude_globs"></a>install_exclude_globs | A list of globs for files to exclude during installation. | List of strings | optional | `[]` |
| <a id="pycross_wheel_library-python_version"></a>python_version | The python version required for this wheel ('PY2' or 'PY3') | String | optional | `""` |
| <a id="pycross_wheel_library-wheel"></a>wheel | The wheel file. | <a href="https://bazel.build/concepts/labels">Label</a> | required | |

@@ -207,6 +205,33 @@ Information about a Python wheel.
| <a id="PycrossWheelInfo-wheel_file"></a>wheel_file | File: The wheel file itself. |


<a id="package_annotation"></a>

## package_annotation

<pre>
package_annotation(<a href="#package_annotation-always_build">always_build</a>, <a href="#package_annotation-build_dependencies">build_dependencies</a>, <a href="#package_annotation-build_target">build_target</a>, <a href="#package_annotation-ignore_dependencies">ignore_dependencies</a>,
<a href="#package_annotation-install_exclude_globs">install_exclude_globs</a>)
</pre>

Annotations to apply to individual packages.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="package_annotation-always_build"></a>always_build | If True, don't use pre-build wheels for this package. | `False` |
| <a id="package_annotation-build_dependencies"></a>build_dependencies | A list of additional package keys (name or name@version) to use when building this package from source. | `[]` |
| <a id="package_annotation-build_target"></a>build_target | An optional override build target to use when and if this package needs to be built from source. | `None` |
| <a id="package_annotation-ignore_dependencies"></a>ignore_dependencies | A list of package keys (name or name@version) to drop from this package's set of declared dependencies. | `[]` |
| <a id="package_annotation-install_exclude_globs"></a>install_exclude_globs | A list of globs for files to exclude during installation. | `[]` |

**RETURNS**

str: A json encoded string of the provided content.


<a id="pypi_file"></a>

## pypi_file
65 changes: 33 additions & 32 deletions e2e/workspace/WORKSPACE
Original file line number Diff line number Diff line change
@@ -89,6 +89,7 @@ load("@rules_pycross//pycross:repositories.bzl", "rules_pycross_dependencies")

rules_pycross_dependencies(python_interpreter)

load("@rules_pycross//pycross:defs.bzl", "package_annotation")
load("@rules_pycross//pycross:workspace.bzl", "lock_repo_model_pdm", "lock_repo_model_poetry", "pycross_lock_file_repo", "pycross_lock_repo", "pycross_register_for_python_toolchains")

pycross_register_for_python_toolchains(
@@ -106,12 +107,22 @@ load("@smoke_environments//:defs.bzl", "environments")
# lock_repo with Poetry
pycross_lock_repo(
name = "poetry_lock_repo",
always_build_packages = [
"regex",
"zstandard",
],
build_target_overrides = {
"zstandard": "@//pdm:zstandard_build",
annotations = {
"regex": package_annotation(
always_build = True,
build_dependencies = [
"setuptools",
"wheel",
],
),
"zstandard": package_annotation(
always_build = True,
build_dependencies = [
"setuptools",
"wheel",
],
build_target = "@//pdm:zstandard_build",
),
},
default_alias_single_version = True,
local_wheels = [
@@ -121,16 +132,6 @@ pycross_lock_repo(
lock_file = "@//:poetry.lock",
project_file = "@//:pyproject.toml",
),
package_build_dependencies = {
"regex": [
"setuptools",
"wheel",
],
"zstandard": [
"setuptools",
"wheel",
],
},
target_environments = environments,
)

@@ -141,28 +142,28 @@ poetry_lock_install_deps()
# lock_repo with PDM
pycross_lock_repo(
name = "pdm_lock_repo",
always_build_packages = [
"regex",
"zstandard",
],
build_target_overrides = {
"zstandard": "@//pdm:zstandard_build",
annotations = {
"regex": package_annotation(
always_build = True,
build_dependencies = [
"setuptools",
"wheel",
],
),
"zstandard": package_annotation(
always_build = True,
build_dependencies = [
"setuptools",
"wheel",
],
build_target = "@//pdm:zstandard_build",
),
},
default_alias_single_version = True,
lock_model = lock_repo_model_pdm(
lock_file = "@//:pdm.lock",
project_file = "@//:pyproject.toml",
),
package_build_dependencies = {
"regex": [
"setuptools",
"wheel",
],
"zstandard": [
"setuptools",
"wheel",
],
},
target_environments = environments,
)

33 changes: 17 additions & 16 deletions e2e/workspace/testlib.bzl
Original file line number Diff line number Diff line change
@@ -4,6 +4,7 @@ load("@aspect_bazel_lib//lib:write_source_files.bzl", "write_source_files")
load("@bazel_skylib//rules:write_file.bzl", "write_file")
load(
"@rules_pycross//pycross:defs.bzl",
"package_annotation",
"pycross_lock_file",
"pycross_wheel_build",
)
@@ -45,22 +46,22 @@ def setup_test_targets(lock_name, lock_model):
lock_model_file = lock_model,
target_environments = ["@smoke_environments//:environments"],
default_alias_single_version = True,
always_build_packages = [
"regex",
"zstandard",
],
build_target_overrides = {
"zstandard": "@@//{}:zstandard_build".format(native.package_name()),
},
package_build_dependencies = {
"regex": [
"setuptools",
"wheel",
],
"zstandard": [
"setuptools",
"wheel",
],
annotations = {
"regex": package_annotation(
always_build = True,
build_dependencies = [
"setuptools",
"wheel",
],
),
"zstandard": package_annotation(
always_build = True,
build_target = "@@//{}:zstandard_build".format(native.package_name()),
build_dependencies = [
"setuptools",
"wheel",
],
),
},
out = "updated_lock.bzl",
)
1 change: 1 addition & 0 deletions examples/bzlmod/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -11,6 +11,7 @@ py_binary(
name = "ipython",
srcs = ["ipython.py"],
deps = [
"@pdm_deps//:amqp-mock",
"@pdm_deps//:ipython",
],
)
9 changes: 9 additions & 0 deletions examples/bzlmod/MODULE.bazel
Original file line number Diff line number Diff line change
@@ -51,6 +51,15 @@ lock_import.package(
repo = "pdm_deps",
)

# Specify per-package overrides. Here we say that we want certain files excluded during installation.
lock_import.package(
name = "amqp-mock",
install_exclude_globs = [
"tests/**",
],
repo = "pdm_deps",
)

# The actual repos are loaded from the lock_repos extension.
lock_repos = use_extension("@rules_pycross//pycross/extensions:lock_repos.bzl", "lock_repos")
use_repo(lock_repos, "pdm_deps", "poetry_deps")
Loading