-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not create runfile trees unnecessarily when building without the bytes #18580
Comments
After spending a lot of time investigating this, I don't see a way to fix this issue that doesn't result in an uncomfortable amount of complexity to be added. The "build without the bytes" machinery only really works for remote actions, but runfiles are an internal (in-process) action, and tracking an "executed-but-not-yet-materialized" state for such an action is not a simple change. It would potentially be worth doing if extended to other in-process actions ( Instead, I think for now we should simply remove
I acknowledge that this leaves us without a mechanism to selectively decide which targets to materialize runfile symlink trees for, but again, I'd like to see evidence that this is necessary. (Note that it will remain the case that runfile symlink trees are created just-in-time whenever required by a local action or a |
…d_minimal. This is arguably not a "build without the bytes" feature; runfile symlink trees are always local (in-process) outputs and never produced remotely. Omitting the flag from the expansion avoids discarding most of the analysis phase when flipping between --remote_download_toplevel and --remote_download_minimal. It remains possible to disable the creation of runfile symlink trees by manually setting --nobuild_runfile_links. See also #18580. RELNOTES: --remote_download_minimal no longer implies --nobuild_runfile_links. PiperOrigin-RevId: 561655765 Change-Id: I1ec0ca3e493bccad0feb666987fef04d9c528b12
Currently,
--remote_download_toplevel
causes all runfiles symlink trees to be created (not just the top-level ones), while--remote_download_minimal
implies--nobuild_runfile_symlinks
and activates a separate code path (seeRunfilesTreeUpdater
) to create input runfiles trees just before local action execution.Instead, we should:
RemoteOutputService#canCreateSymlinks
and provide an emptyRemoteOutputService#createSymlinks
implementation; this ensures the runfiles tree is never materialized as a direct result of running theSymlinkTreeAction
.SpawnRunner#prefetchInputs
create the runfiles tree before local action execution (as is already the case for other local action inputs).AbstractActionInputPrefetcher#finalizeAction
create the runfiles tree for toplevel targets (as is already the case for other top-level outputs).This would ensure a minimal set of runfile trees is created in all cases, and let us omit
--nobuild_runfile_links
from the expansion of--remote_download_minimal
.The text was updated successfully, but these errors were encountered: