[pull] master from bazelbuild:master #1992
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
Work towards #11820 Fixes #20038 Fixes #23200 Fixes #24613 RELNOTES: Extra targets provided to `ctx.expand_location` now expand to their executable (if any) instead of resulting in an error if they provide a number of files different from one. RELNOTES[INC]: The `--incompatible_locations_prefers_executable` flag has been added and enabled, which makes it so that `ctx.expand_location` expands `$(locations :x)` to the executable of an extra target `:x` if it provides one and the number of files provided by it is not one. Closes #24690. PiperOrigin-RevId: 713453768 Change-Id: I0d6e052bc70deea029554ab722feb544f9597a23
By default, the JDK only reports up to 500 events per `WatchKey` (and thus, per directory). This can be too low in large repos. Work towards #13226 RELNOTES: On Linux, the default limit on the number of `--watchfs` file events per directory has been raised to 10,000 (from 500). If needed, it can be increased further via `--host_jvm_args=-Djdk.nio.file.WatchService.maxEventsPerPoll=<limit>`. Closes #23682. PiperOrigin-RevId: 713570760 Change-Id: Ib3a61ad5d7cfaad4b7bbc25110f854e978e7fa75
The required conversion to `PathFragment` can be performed during execution, which avoids retaining a new `PathFragment` instance per action. Closes #24831. PiperOrigin-RevId: 713577830 Change-Id: I2781ea979a3c97abf5a519acd5d3d6a668dd1f77
New `->`-style switch expressions on enums have built-in handling of exhaustiveness, if a default is omitted the switch will implicitly throw a `MatchException` if the switch has an impossible value. PiperOrigin-RevId: 713785508 Change-Id: I57614847231568d213c02b0f91b5cb181542ad59
…l, Starlark, StarlarkCallable and StarlarkFunction. This is the envisioned alternative to fastcall() where the function parameters aren't stored in intermediate arrays or collections, and instead are directly placed in the right spot in the locals[] array which is then assigned to Frame.locals and thus passed to Eval.execFunctionBody(). PiperOrigin-RevId: 713955817 Change-Id: I7915783d689c85a0c0346ac325d308d3c4a764fd
PiperOrigin-RevId: 713963905 Change-Id: I1d7efddf699cc5c5a43186f65162ac372351f036
This should be a no-op. PiperOrigin-RevId: 713981634 Change-Id: I3df4926f00c92104a9aa5d66950374ad9eb4d4d0
These tests became ineffective with the flip of `--experimental_inprocess_symlink_creation`. Closes #24879. PiperOrigin-RevId: 713986922 Change-Id: I1f7aa6f1eb37f3907e88054b1221267288ef3ec9
... which has kythe 0.0.68. PiperOrigin-RevId: 714012884 Change-Id: I451ccf9804aead1a57e6d7c5c397a4618a80c22d
This fills in a few gaps in the timing profile. Closes #24762. PiperOrigin-RevId: 714028550 Change-Id: I5772b59c5d3b6d73306ea5d8380169465f348f27
The previous doc say it told if it was a directory; probably a copy/paste mistake. PiperOrigin-RevId: 714089602 Change-Id: Ib6f6bee762afdd41f7249f120f4f95a088a3ccef
PiperOrigin-RevId: 714111381 Change-Id: Ie168c15f2d035b2a811ec0e76f1cab04adf4dfa8
These refer to the deprecated `--host_cpu` flag, which is bad to select on anyway. Closes #24898. PiperOrigin-RevId: 714169621 Change-Id: I97c751cd781f0ccbd0a36086fe4f6c927b1a3d8d
…y in BUILD file. Closes #24899. PiperOrigin-RevId: 714169712 Change-Id: Ib0e30ec99fb83118ac29900baaecb1db3d5e0b86
Caused by `String.split` returning an empty array if there is a comment containing an empty comment PiperOrigin-RevId: 714172391 Change-Id: I7e006c5fc37b370d45c41083fe581e3ef9e7e873
The new approach is that the Blaze client tells the server's JVM to create a special file before it throws an OOM; then if the Blaze client is unable to otherwise determine the exit code to use (i.e. gRPC connection broken and the server was unable to write an abrupt exit file), it uses the presence of this file to determine it should use `33`. By having the mechanism be solely in client-land and at the lowest possible level in the JVM, we're no longer at the mercy of the server's app land code working correctly and arbitrary code outside our control not killing JVM before our existing mitigations (like the abrupt exit file) kick in. The new approach as-is improves quality but doesn't improve speed. As an optional future improvement, we can have the invoker of Blaze notice the write of the special file as soon as it happens and immediately use an exit code of `33`. PiperOrigin-RevId: 722716155 Change-Id: I00d0f9435bc812678703cd7caf29e20b6351f185
This was showing up in CPU profiles. PiperOrigin-RevId: 722730253 Change-Id: I1ac18345767bada11347d0d29ef7156330fd76ab
This new attribute contains the original value of the `name` attribute at the instantiation site of the repo rule (e.g., `rctx.original_name` would be `foo` if `rctx.name` is `+ext+foo`). Fixes #24467 Closes #25121. RELNOTES: Added `repository_ctx.original_name`, which contains the original value of the `name` attribute as specified at the repo rule call site. PiperOrigin-RevId: 722731393 Change-Id: I2f7dada0c44b6bd4c0d2622fa1e97223382a8547
For some reason this didn't happen in baa9e55. Also took this chance to delete some really old versions. PiperOrigin-RevId: 722759259 Change-Id: Ia117670142dab2406ba839a951d15218cb287b33
Using collection types that aren't guaranteed to preserve insertion order risks errors that differ across machines or builds. Closes #25175. PiperOrigin-RevId: 722768825 Change-Id: Id84c6b5b4bc9a1e6938de727e07c20ee761cbf3d
With `--incompatible_bazel_test_exec_run_under`, the `--run_under` target should be configured for the execution platform of the test action, not the default execution platform of the test rule. This is still not fully correct if `testing.ExecutionInfo` is used to specify an alternative test exec group, but that's a far more difficult change: it would potentially involve a split transition with one configuration for each exec group. Work towards #23179 Closes #25177. PiperOrigin-RevId: 722802891 Change-Id: Ie168bf496d46645f7bd2ad739859504c65286d20
…dation. This change introduces the `VersionedChangesValidator` class, which is responsible for determining the validity of cache entries based on file system dependencies and their associated versions. The `VersionedChangesValidator` checks if changes in files or directory listings (represented by `VersionedChanges`) invalidate a cached value. It wraps `FileOpMatchMemoizingLookup` for individual file operations and `NestedMatchMemoizingLookup` for nested file system dependencies. *Caching Semantics and `validityHorizon`:* A significant aspect of this change is the detailed explanation of caching behavior in relation to the `validityHorizon` parameter, documented in the `VersionedChangesValidator` class comments. Notably, even with different `validityHorizon` values, as long as they are generated correctly, they yield equivalent results in terms of cached computation validity. This allows sharing versioned changes cache entries between cached values evaluated at different versions, leading to substantial cost savings. *Refactorings:* - Moved `NestedDependencies` instantiation from tests into the `NestedDependencies` class to reduce duplication. - Moved `NestedMatchResult` instantiation from `NestedMatchMemoizingLookup` to its own class and added unit tests. PiperOrigin-RevId: 722817997 Change-Id: Iaee350ed4439352c58300d8a394f97fbca042d14
…urations. Technically strings like "1e2" are not valid for the `time` attribute (that attribute's schema is supposed to be `xs:decimal`), however some JUNIT writers are non-compliant and use scientific e notation and we want Bazel to still work with them. While I'm here, improve the code comments (the existing comment about us having to support values without decimal points like "12" as milliseconds is backwards as noted in #24605 (comment)) and add tests. Fixes #24605. PiperOrigin-RevId: 722963681 Change-Id: I9383cf7435ae7843f07a5abddbcf790f275403d0
bazelbuild/java_tools#95 Closes #25048. PiperOrigin-RevId: 722999495 Change-Id: I8b7e106ff4f6ee32363b4047841d10e706da9c6d
*** Reason for rollback *** This seems to cause an issue. See b/394174527#comment3 *** Original change description *** Move the postponing-invalidation logic from WorkerLifecycleManager to WorkerPoolImpl. All idle workers in WokerPoolImpl can be killed, so the postponing validation logic should be placed inside the WorkerPool instead of WorkerLifecycleManager. PiperOrigin-RevId: 723080609 Change-Id: I51888e1e9ebd1d818e16326b97e5fd6b122f6e8f
…ke variables for the aspect and the base rule separately. - Updates the existing aspect-specific `ctx.var` method: it now returns only aspect-related variables. - Adds a new aspect-specific `ctx.rule.var` method, which returns only rule-related variables. - Removes the ability of native rules to call `RuleContext.initConfigurationMakeVariableContext`, as it was unused and adding complexity. Rollback of 50e8136 with additional fixes. RELNOTES: Adds `ctx.rule.var` to allow aspects to get rule-specific variables, and removes rule-specific variables from an aspect's `ctx.var` dict. PiperOrigin-RevId: 723141782 Change-Id: I9db4262eca3428d9a4e93278c6498089293e2e90
This change introduces a `WriteStatus` interface extending
`ListenableFuture<Void>` specifically designed for tracking write
operation success or failure. The key optimization is a "sparse
aggregation" mode that significantly reduces in-flight memory usage,
especially when the `FrontierSerializer` is active.
**Motivation:**
Profiling revealed that write status listeners were a major
contributor to memory consumption in certain builds when using the
`FrontierSerializer`. In this scenario, write statuses are continuously
aggregated without being dropped.
**Optimization:**
`WriteStatus` supports a sparse aggregation mode where only the *first*
listener added to a future is retained. Subsequent listeners are
ignored. This preserves the property that the top-level status reflects
every individual status because, with `FrontierSerializer`, all statuses
are eventually aggregated into a single future.
**Implementation:**
The `WriteStatuses` utility class provides factory methods and
implementations for:
- `ImmediateWriteStatus`: A singleton representing an immediately successful
write.
- `ImmediateFailedWriteStatus`: Represents an immediately failed write.
- `SettableWriteStatus`: Analogous to `SettableFuture`, allowing manual
success/failure marking.
- `SparseAggregateWriteStatus`: Aggregates multiple `WriteStatus` instances
sparsely.
- `AggregateWriteStatus`: A general-purpose aggregator (non-sparse).
These will replace `ListenableFuture<Void>` when signaling write status.
PiperOrigin-RevId: 723172312
Change-Id: If093e22d38e91429d5860e7f8434a3297956ce5c
This allows sparse futures aggregation in serialization to save memory. PiperOrigin-RevId: 723173604 Change-Id: Idefe48fbee99f457d600032793c72741d0d0576c
*** Reason for rollback *** performance regression b/392290938 *** Original change description *** Remove fastcall() from StarlarkCallable.java and Starlark.java PiperOrigin-RevId: 723363896 Change-Id: Ic0fd4f5642b106d1e2d3843555f0ebc1fdf7b140
*** Reason for rollback *** performance regression b/392290938 *** Original change description *** Remove fastcall() and related code from BuiltinFunction PiperOrigin-RevId: 723375619 Change-Id: Ib1dd60269a09472436fe3a03f3fab80402865840
*** Reason for rollback *** performance regression b/392290938 *** Original change description *** Remove use of fastcall() from Eval.java PiperOrigin-RevId: 723396402 Change-Id: Icc69adbd41a5148349464544b918d2d7190755e7
There is no need anymore to allow passing in your own logger. PiperOrigin-RevId: 723405888 Change-Id: I8a3ae9c179a4f4a0b9e44a757ee3b2e710545966
This is a temporary fix for a performance regression that the use of BuiltinFunction.ArgumentProcessor caused. PiperOrigin-RevId: 723413626 Change-Id: I395052c8e05047223dc8b342fbe5c7d6b57369cc
PiperOrigin-RevId: 723455636 Change-Id: I959e5bef17038e8a02b24ae2aed65300345ef901
…d as a symlink to another. This will be useful in upcoming changes extending the concept of lazy materialization to non-remote artifacts. PiperOrigin-RevId: 723477378 Change-Id: I704f13782c0dbfeb65f9e13e2dbaede745ec2202
Bazel doesn't create "sub-runfiles". Closes #25159. PiperOrigin-RevId: 723480332 Change-Id: I11813e1b60d058f38e60e5ae7cacea0585ca08a3
PiperOrigin-RevId: 723484432 Change-Id: I8ff8e47e87400c6d4c2629de6353204784037714
Most of the linking code is in Starlark, including a large chunk of LTO support. What's not in Starlark: - conversion of LibraryToLink into LegacyLinkerInputs (causes a regression) - conversion of collectLibrariesToLink (causes a regression) - compilation of LTO artifacts (needs rewrite of C++ compilation support first) PiperOrigin-RevId: 723497031 Change-Id: I9cd75a9c11de9eb82b277757267bcaeaecfd7034
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )