Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unresolved symlink to absolute path in remote action outputs is not supported #16290

Closed
tjgq opened this issue Sep 16, 2022 · 6 comments
Closed
Labels
P1 I'll work on this now. (Assignee required) team-Remote-Exec Issues and PRs for the Execution (Remote) team

Comments

@tjgq
Copy link
Contributor

tjgq commented Sep 16, 2022

When a remote action produces an undeclared symlink (i.e., a ctx.actions.declare_symlink) pointing to an absolute path, an exception is thrown here: https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/remote/RemoteExecutionService.java;l=745;drc=bce7db055dd702e8a101f975f8585e0dfeb18a67

We will also have to fix #16289 before adding a regression test for this issue.

This issue may be a blocker for #10298.

Note that this is separate from #13355, which concerns any kind of symlink in action outputs (resolved or unresolved, relative or absolute, dangling or non-dangling) and only applies when building without the bytes.

@tjgq
Copy link
Contributor Author

tjgq commented Sep 16, 2022

While we're at it, we should also remove the limitation in https://cs.opensource.google/bazel/bazel/+/master:src/main/java/com/google/devtools/build/lib/remote/RemoteActionFileSystem.java;l=222;drc=79b5b5ddaa3919db48b9143e0afdd4d2cddfdb74 (which AIUI would only apply when a remotely produced declare_symlink is an input to a local action).

tjgq added a commit to tjgq/bazel that referenced this issue Sep 16, 2022
The tests don't currently pass due to bazelbuild#16290 and bazelbuild#16289. Making them pass might
be considered a blocker for bazelbuild#10298 (declaring unresolved symlinks stable).
@lberki
Copy link
Contributor

lberki commented Sep 16, 2022

I'm not too worried about --experimental_remote_download_outputs=minimal because it's experimental so it's fine for it to be slightly broken, but this indeed needs to be fixed.

I think what's more important than fixing the mock local executor is exercising this functionality with the real RBE so #16289 should not be a blocker for this. It would be nice, but I could live without it.

@lberki lberki added release blocker P1 I'll work on this now. (Assignee required) team-Remote-Exec Issues and PRs for the Execution (Remote) team labels Sep 16, 2022
@brentleyjones
Copy link
Contributor

I'm not too worried about --experimental_remote_download_outputs=minimal because it's experimental so it's fine for it to be slightly broken.

While this feature is listed as experimental, I can speak for the community and say many, many projects depend on it.

@brentleyjones
Copy link
Contributor

Actually, it's not experimental anymore: https://bazel.build/reference/command-line-reference#flag--remote_download_outputs

@lberki
Copy link
Contributor

lberki commented Sep 16, 2022

@brentleyjones that's unfortunate, I didn't realize we promoted that flag to production, thanks for pointing it out

@yazanabadi
Copy link

عندما ينتج عن إجراء عن بُعد ارتباط رمزي غير معلن (على سبيل المثال ، ctx.actions.declare_symlink) يشير إلى مسار مطلق ، يتم طرح استثناء هنا: https://cs.opensource.google/bazel/bazel/+/master:src/ main / java / com / google / devtools / build / lib / remote / RemoteExecutionService.java؛ l = 745؛ drc = bce7db055dd702e8a101f975f8585e0dfeb18a67

سيتعين علينا أيضًا إصلاح # 16289 قبل إضافة اختبار انحدار لهذه المشكلة.

قد تكون هذه المشكلة مانعًا لـ # 10298 .

لاحظ أن هذا منفصل عن # 13355 ، والذي يتعلق بأي نوع من مخرجات الارتباط الرمزي في العمل (تم حلها أو لم يتم حلها ، نسبيًا أو مطلقًا ، متدليًا أو غير متدلي) ولا يتم تطبيقه إلا عند الإنشاء بدون وحدات البايت.

@meteorcloudy meteorcloudy added this to the 6.0.0 release blockers milestone Sep 27, 2022
aiuto pushed a commit to aiuto/bazel that referenced this issue Oct 12, 2022
Fixes bazelbuild#16290.

PiperOrigin-RevId: 478048404
Change-Id: Ibc5b7f681a520dab6945cb18d976440c9f2e62a7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 I'll work on this now. (Assignee required) team-Remote-Exec Issues and PRs for the Execution (Remote) team
Projects
None yet
Development

No branches or pull requests

5 participants