Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Starlarkify all repository rules #18285

Closed
5 of 6 tasks
Wyverald opened this issue May 2, 2023 · 8 comments
Closed
5 of 6 tasks

Starlarkify all repository rules #18285

Wyverald opened this issue May 2, 2023 · 8 comments
Labels
area-Bzlmod Bzlmod-specific PRs, issues, and feature requests P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request
Milestone
Mainline issues t...

Comments

@Wyverald
Copy link
Member

Wyverald commented May 2, 2023 •
edited
Loading

We originally wanted to allow native repo rule usage in module extensions (#15412), but that turned out to be surprisingly hard to implement.

Instead, we could just go down the other route: Starlarkify all the repo rules!
@Wyverald Wyverald mentioned this issue May 2, 2023
Closed
@Wyverald Wyverald added type: feature request P1 I'll work on this now. (Assignee required) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. area-Bzlmod Bzlmod-specific PRs, issues, and feature requests labels May 2, 2023
@Wyverald Wyverald self-assigned this May 2, 2023
@Wyverald
Copy link
Member Author

Wyverald commented May 2, 2023

cc @ted-xie @ahumesky @kotlaja

@fmeum
Copy link
Collaborator

fmeum commented May 2, 2023

@Wyverald Starlarkifying local_config_platform would get us halfway to #8766 (comment), so I think it's worth doing.

@ahumesky
Copy link
Contributor

ahumesky commented May 2, 2023

A starkark version of android_ndk_repository is mostly complete here: https://github.com/bazelbuild/rules_android_ndk

@katre katre mentioned this issue May 2, 2023
Closed
@Wyverald
Copy link
Member Author

Wyverald commented May 4, 2023

See https://docs.google.com/document/d/17RZKMuMjAIgNfFdrsLhqNsMTQDSS-BBA-S-fCSBPV7M/edit# for the design to address the missing API.

@Wyverald Wyverald added this to Bzlmod Jul 12, 2023
@Wyverald Wyverald moved this to Todo in Bzlmod Jul 12, 2023
@Wyverald Wyverald added this to the 7.0.0 branch cut milestone Jul 13, 2023
@Wyverald Wyverald added P2 We'll consider working on this in future. (Assignee optional) and removed P1 I'll work on this now. (Assignee required) labels Jul 17, 2023
@meteorcloudy meteorcloudy removed this from the 7.0.0 branch cut milestone Sep 19, 2023
@Wyverald Wyverald mentioned this issue Nov 13, 2023
Closed
@Wyverald Wyverald added this to the 7.1.0 release blockers milestone Nov 16, 2023
@iancha1992 iancha1992 removed this from the 7.1.0 release blockers milestone Nov 29, 2023
@iancha1992
Copy link
Member

@bazel-io fork 7.1.0

@bazel-io bazel-io mentioned this issue Nov 29, 2023
Closed
@keertk keertk added this to the 7.1.0 release blockers milestone Nov 29, 2023
@meteorcloudy meteorcloudy mentioned this issue Jan 24, 2024
Closed
@Nitikonkon2534 Nitikonkon2534 mentioned this issue Jan 31, 2024
Open
@kklochkov kklochkov mentioned this issue Apr 16, 2024
Merged
Wyverald added a commit that referenced this issue Apr 16, 2024
@Wyverald
Add new flag --incompatible_disable_native_repo_rules
e189c49 
This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod.

Work towards #18285.
@aaronmondal
Copy link

@Wyverald I tried to use the local repo rules in an extension with 8.0.0-pre.20240401.3 which contains 6f254ce, but it doesn't seem to work.

In extension.bzl I have something like this (same issue for new_local_repository):

load("@bazel_tools//tools/build_defs/repo:local.bzl", "local_repository")

def _extension_impl(ctx):
    local_repository(name = "foo", path = "../../")
    
myext = module_extension(
    implementation = _extension_impl,
)

In MODULE.bazel something like this:

ext = use_extension("//:extension.bzl", "myext")

use_repo(ext, "foo")

It raises this error telling me that the local.bzl file can't be found. Is this intended behavior because it's a prerelease or could it be a bug?

MODULE.bazel:xx:yy:
  at .../extensions.bzl:6:6: cannot load '@@bazel_tools//tools/build_defs/repo:local.bzl': no such file:
  at .../extensions.bzl:6:6: cannot load '@@bazel_tools//tools/build_defs/repo:local.bzl': no such file

@aaronmondal aaronmondal mentioned this issue Apr 16, 2024
Open
@fmeum
Copy link
Collaborator

fmeum commented Apr 16, 2024

@aaronmondal I can't reproduce this with USE_BAZEL_VERSION=8.0.0-pre.20240401.3. Both in WORKSPACE and MODULE.bazel load("@bazel_tools//tools/build_defs/repo:local.bzl", "local_repository") works just fine.

@aaronmondal
Copy link

It looks like i messed something up. I don't know what it was but your right it works. Thanks for checking, I was about to defer this and now it looks like we can get LLVM working with bzlmod after all ❤️

Wyverald added a commit that referenced this issue Apr 17, 2024
@Wyverald
Add new flag --incompatible_disable_native_repo_rules
46b984a 
This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod.

Work towards #18285.
Wyverald added a commit that referenced this issue Apr 19, 2024
@Wyverald
Add new flag --incompatible_disable_native_repo_rules
7958e88 
This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod.

Work towards #18285.
Wyverald added a commit that referenced this issue Apr 19, 2024
@Wyverald
Add new flag --incompatible_disable_native_repo_rules
000d1a0 
This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod.

Work towards #18285.
copybara-service bot pushed a commit that referenced this issue Apr 19, 2024
@Wyverald @copybara-github
Add new flag --incompatible_disable_native_repo_rules
09d50f9 
This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod.

Work towards #18285.

Closes #21913.

PiperOrigin-RevId: 626410163
Change-Id: I4981cd8d99fc87d0d0b66991e563941478b40f12
@github-project-automation github-project-automation bot moved this from In Progress to Done in Bzlmod Apr 22, 2024
@Wyverald Wyverald mentioned this issue Apr 22, 2024
Open
Wyverald added a commit that referenced this issue Apr 25, 2024
@Wyverald
Local repo rules in Starlark
dd192e7 
Added `local_repository` and `new_local_repository` as Starlark repo rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They're drop-in replacements for their native counterparts.

Work towards #18285.

RELNOTES: The `local_repository` and `new_local_repository` repository rules are now available as Starlark rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They are drop-in replacements for their native counterparts, and can be used in module extensions.

Closes #21681.

PiperOrigin-RevId: 615926923
Change-Id: I0cc7355b011751da23f2a3aa189f120ef177e0fe
Wyverald added a commit that referenced this issue Apr 25, 2024
@Wyverald
Add missing TOC entry to Starlark local repo rules
41a3ba1 
These were added in 6f254ce but never added to the table of contents.

Work towards #18285.

PiperOrigin-RevId: 617904579
Change-Id: I062f7d2fb4bcd441db566e4d00ed9d5d9af43886
Wyverald added a commit that referenced this issue Apr 25, 2024
@Wyverald
Use the host_platform from @platforms in Bazel
1bb00bc 
* Upgrade to `platforms` 0.0.9
* `--host_platform` now defaults to `@bazel_tools//tools:host_platform`, which is an alias of `@platforms//host`
* `local_config_platform` (the repo rule) now just outputs a thin wrapper; `@local_config_platform//:host` is an alias of `@platforms//host`, and `@local_config_platform//:constraints.bzl` re-exports `@platforms//host:constraints.bzl`
* Removed all test mocks of `local_config_platform`. A follow-up will guard prod usage behind a flag.

Work towards #8766 and #18285.

PiperOrigin-RevId: 620316973
Change-Id: I6593f62569f31faee69e88a520a0f7f42009e05d
Wyverald added a commit that referenced this issue Apr 25, 2024
@Wyverald
Forcibly use the bundled version of @platforms for loading the host…
67c3255 
… platform repo rule

#21877 happened because, even though we define `@platforms` to be the bundled version shipped with Bazel (which is 0.0.9 on HEAD), people tend to define their own versions of `@platforms` in WORKSPACE. If this ends up being a lower version (very likely), the subsequent `load` to use the `host_platform_repo` repo rule (which is a Starlark version of `local_config_platform`) will fail because that repo rule was only introduced in `platforms` version 0.0.9.

To resolve this conundrum, we define a new repo `@internal_platforms_do_not_use` that _also_ points to the bundled version of `@platforms`. Nobody is likely to define this repo, so we can fairly reliably get the Starlark repo rule, and also still allow people to override the actual `@platforms` targets they use.

This only affects WORKSPACE users; if `--enable_bzlmod` is set, the `@host_platform` seen by `@bazel_tools` will come from the module extension instead.

Work towards #18285.

Fixes #21877.

Closes #21902.

PiperOrigin-RevId: 622238362
Change-Id: I6335ab1bc26ba9ef54473d8a21fc5a8510a3ace5
Wyverald added a commit that referenced this issue Apr 25, 2024
@Wyverald
Add new flag --incompatible_disable_native_repo_rules
2d50139 
This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod.

Work towards #18285.

Closes #21913.

PiperOrigin-RevId: 626410163
Change-Id: I4981cd8d99fc87d0d0b66991e563941478b40f12
Wyverald added a commit that referenced this issue Apr 29, 2024
@Wyverald
Local repo rules in Starlark
bad6f4d 
Added `local_repository` and `new_local_repository` as Starlark repo rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They're drop-in replacements for their native counterparts.

Work towards #18285.

RELNOTES: The `local_repository` and `new_local_repository` repository rules are now available as Starlark rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They are drop-in replacements for their native counterparts, and can be used in module extensions.

Closes #21681.

PiperOrigin-RevId: 615926923
Change-Id: I0cc7355b011751da23f2a3aa189f120ef177e0fe
Wyverald added a commit that referenced this issue Apr 29, 2024
@Wyverald
Add missing TOC entry to Starlark local repo rules
b17c1fa 
These were added in 6f254ce but never added to the table of contents.

Work towards #18285.

PiperOrigin-RevId: 617904579
Change-Id: I062f7d2fb4bcd441db566e4d00ed9d5d9af43886
Wyverald added a commit that referenced this issue Apr 29, 2024
@Wyverald
Use the host_platform from @platforms in Bazel
eda321f 
* Upgrade to `platforms` 0.0.9
* `--host_platform` now defaults to `@bazel_tools//tools:host_platform`, which is an alias of `@platforms//host`
* `local_config_platform` (the repo rule) now just outputs a thin wrapper; `@local_config_platform//:host` is an alias of `@platforms//host`, and `@local_config_platform//:constraints.bzl` re-exports `@platforms//host:constraints.bzl`
* Removed all test mocks of `local_config_platform`. A follow-up will guard prod usage behind a flag.

Work towards #8766 and #18285.

PiperOrigin-RevId: 620316973
Change-Id: I6593f62569f31faee69e88a520a0f7f42009e05d
Wyverald added a commit that referenced this issue Apr 29, 2024
@Wyverald
Forcibly use the bundled version of @platforms for loading the host…
09df65a 
… platform repo rule

#21877 happened because, even though we define `@platforms` to be the bundled version shipped with Bazel (which is 0.0.9 on HEAD), people tend to define their own versions of `@platforms` in WORKSPACE. If this ends up being a lower version (very likely), the subsequent `load` to use the `host_platform_repo` repo rule (which is a Starlark version of `local_config_platform`) will fail because that repo rule was only introduced in `platforms` version 0.0.9.

To resolve this conundrum, we define a new repo `@internal_platforms_do_not_use` that _also_ points to the bundled version of `@platforms`. Nobody is likely to define this repo, so we can fairly reliably get the Starlark repo rule, and also still allow people to override the actual `@platforms` targets they use.

This only affects WORKSPACE users; if `--enable_bzlmod` is set, the `@host_platform` seen by `@bazel_tools` will come from the module extension instead.

Work towards #18285.

Fixes #21877.

Closes #21902.

PiperOrigin-RevId: 622238362
Change-Id: I6335ab1bc26ba9ef54473d8a21fc5a8510a3ace5
Wyverald added a commit that referenced this issue Apr 29, 2024
@Wyverald
Add new flag --incompatible_disable_native_repo_rules
6339bbc 
This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod.

Work towards #18285.

Closes #21913.

PiperOrigin-RevId: 626410163
Change-Id: I4981cd8d99fc87d0d0b66991e563941478b40f12
Wyverald added a commit that referenced this issue Apr 30, 2024
@Wyverald
Local repo rules in Starlark
e565fa5 
Added `local_repository` and `new_local_repository` as Starlark repo rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They're drop-in replacements for their native counterparts.

Work towards #18285.

RELNOTES: The `local_repository` and `new_local_repository` repository rules are now available as Starlark rules under `@bazel_tools//tools/build_defs/repo:local.bzl`. They are drop-in replacements for their native counterparts, and can be used in module extensions.

Closes #21681.

PiperOrigin-RevId: 615926923
Change-Id: I0cc7355b011751da23f2a3aa189f120ef177e0fe
Wyverald added a commit that referenced this issue Apr 30, 2024
@Wyverald
Add missing TOC entry to Starlark local repo rules
bf641ce 
These were added in 6f254ce but never added to the table of contents.

Work towards #18285.

PiperOrigin-RevId: 617904579
Change-Id: I062f7d2fb4bcd441db566e4d00ed9d5d9af43886
Wyverald added a commit that referenced this issue Apr 30, 2024
@Wyverald
Use the host_platform from @platforms in Bazel
0f5b013 
* Upgrade to `platforms` 0.0.9
* `--host_platform` now defaults to `@bazel_tools//tools:host_platform`, which is an alias of `@platforms//host`
* `local_config_platform` (the repo rule) now just outputs a thin wrapper; `@local_config_platform//:host` is an alias of `@platforms//host`, and `@local_config_platform//:constraints.bzl` re-exports `@platforms//host:constraints.bzl`
* Removed all test mocks of `local_config_platform`. A follow-up will guard prod usage behind a flag.

Work towards #8766 and #18285.

PiperOrigin-RevId: 620316973
Change-Id: I6593f62569f31faee69e88a520a0f7f42009e05d
Wyverald added a commit that referenced this issue Apr 30, 2024
@Wyverald
Forcibly use the bundled version of @platforms for loading the host…
e28c47d 
… platform repo rule

#21877 happened because, even though we define `@platforms` to be the bundled version shipped with Bazel (which is 0.0.9 on HEAD), people tend to define their own versions of `@platforms` in WORKSPACE. If this ends up being a lower version (very likely), the subsequent `load` to use the `host_platform_repo` repo rule (which is a Starlark version of `local_config_platform`) will fail because that repo rule was only introduced in `platforms` version 0.0.9.

To resolve this conundrum, we define a new repo `@internal_platforms_do_not_use` that _also_ points to the bundled version of `@platforms`. Nobody is likely to define this repo, so we can fairly reliably get the Starlark repo rule, and also still allow people to override the actual `@platforms` targets they use.

This only affects WORKSPACE users; if `--enable_bzlmod` is set, the `@host_platform` seen by `@bazel_tools` will come from the module extension instead.

Work towards #18285.

Fixes #21877.

Closes #21902.

PiperOrigin-RevId: 622238362
Change-Id: I6335ab1bc26ba9ef54473d8a21fc5a8510a3ace5
Wyverald added a commit that referenced this issue Apr 30, 2024
@Wyverald
Add new flag --incompatible_disable_native_repo_rules
3aab422 
This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod.

Work towards #18285.

Closes #21913.

PiperOrigin-RevId: 626410163
Change-Id: I4981cd8d99fc87d0d0b66991e563941478b40f12
Kila2 pushed a commit to Kila2/bazel that referenced this issue May 13, 2024
@Wyverald @Kila2
Forcibly use the bundled version of @platforms for loading the host…
f8116d2 
… platform repo rule

bazelbuild#21877 happened because, even though we define `@platforms` to be the bundled version shipped with Bazel (which is 0.0.9 on HEAD), people tend to define their own versions of `@platforms` in WORKSPACE. If this ends up being a lower version (very likely), the subsequent `load` to use the `host_platform_repo` repo rule (which is a Starlark version of `local_config_platform`) will fail because that repo rule was only introduced in `platforms` version 0.0.9.

To resolve this conundrum, we define a new repo `@internal_platforms_do_not_use` that _also_ points to the bundled version of `@platforms`. Nobody is likely to define this repo, so we can fairly reliably get the Starlark repo rule, and also still allow people to override the actual `@platforms` targets they use.

This only affects WORKSPACE users; if `--enable_bzlmod` is set, the `@host_platform` seen by `@bazel_tools` will come from the module extension instead.

Work towards bazelbuild#18285.

Fixes bazelbuild#21877.

Closes bazelbuild#21902.

PiperOrigin-RevId: 622238362
Change-Id: I6335ab1bc26ba9ef54473d8a21fc5a8510a3ace5
Kila2 pushed a commit to Kila2/bazel that referenced this issue May 13, 2024
@Wyverald @Kila2
Add new flag --incompatible_disable_native_repo_rules
a7395da 
This new flag defaults to `false`. When set to `true`, native repo rules cannot be used in WORKSPACE; their Starlark counterparts must be used. Native repo rules already can't be used in Bzlmod.

Work towards bazelbuild#18285.

Closes bazelbuild#21913.

PiperOrigin-RevId: 626410163
Change-Id: I4981cd8d99fc87d0d0b66991e563941478b40f12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-Bzlmod Bzlmod-specific PRs, issues, and feature requests P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request
Projects
Bzlmod
Archived in project
Milestone
Mainline issues targeted for 7.2.0
Development

No branches or pull requests
8 participants
@Wyverald @meteorcloudy @fmeum @ahumesky @aaronmondal @keertk @iancha1992 and others