-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[v0.13] OTEL socket mount requires new volumes when containerd is run in separate container #4764
Comments
It should be fixed with #4619 Can you try with |
Hey @crazy-max
PS: We get the same failure with |
hum these tags exist: |
We are using buildkit in the context of a Tiltfile extension I've verified in this setup that the current |
@brettmorien Are you using some kind of configuration where containerd is running in a different container/rootfs than buildkit? |
We are using this extension for Tilt for local development: https://github.com/tilt-dev/tilt-extensions/tree/master/kubectl_build How that's translated through all the layers of stuff is a container spec that looks like:
|
@brettmorien Looks like that setup is indeed running container and buildkit in separate containers and setting up some volume mounts in between them. https://github.com/vmware-archive/buildkit-cli-for-kubectl/blob/1db649b1f50268d857d0cfd36335800c72d2cf50/pkg/driver/kubernetes/manifest/manifest.go#L178-L209 With the OTEL socket being in @georgethebeatle is your setup somewhat similar to @brettmorien 's ? |
@tonistiigi we are using the buildkit cli for kubectl configuration of |
FWIW I tried the same build with a patched version of corev1.Volume{
Name: "run-buildkit",
VolumeSource: corev1.VolumeSource{
HostPath: &corev1.HostPathVolumeSource{
Path: "/var/run/buildkit",
Type: &hostPathDirectory,
},
},
},
//...
corev1.VolumeMount{
Name: "run-buildkit",
MountPath: "/run/buildkit",
MountPropagation: &mountPropagationBidirectional,
}, Then I started getting this error:
When I shell into the kind cluster's docker container I cannot find an otel sock anywhere ( Another interesting finding is that when I try to run the same build against a remote kubernetes cluster (with the original untampered version of the buildkit cli for kubectl) the build runs successfully. When I shell in the buildkit pod I can see that |
@tonistiigi are you sure that the cause is running containerd in separate container? If so wouldn't it also reproduce on remote clusters? For us it only happens on kind. |
Closing since this doesn't appear to be directly a buildkit error and related to the outdated tool buildkit-cli-for-kubectl configuring volumes for deploying buildkit |
Steps to reproduce
What is expected to happen
The kbld command succeeds
What actually happens
The command fails with the follwing output:
Debugging
We were able to bisect that this PR is introducing the failure
The text was updated successfully, but these errors were encountered: