Is there a way to --strip-components with ADD

[ghost]

A server is hosting a rootfs-wrapper.tar.xz file which instead of having this structure:

/bin
/sbin
/usr
...

is having a wrapper directory

/wrapper/bin
/wrapper/sbin
/wraper/usr
...

With regular shell we can do:

curl -O http://site/rootfs-wrapper.tar.xz
tar -xf rootfs.tar.xz --strip-components=1 -C .

How do we do that with scratch docker where we don't have access to tar and curl and viable option is to use ADD command

FROM scratch
ADD http://site/rootfs-wrapper.tar.xz /
CMD ["/bin/sh"]

it builds fine with docker build -t simple-build ., but fails to run docker run -it simple-build gives:

Error response from daemon: OCI runtime create failed: container_linux.go:296: starting container process caused "exec: \"/bin/sh\": stat /bin/sh: no such file or directory": unknown.

If we use rootfs.tar.xz without wrapper/ then docker run works without modifying the docker.
I tried setting CMD ["/wrapper/bin/sh"] in docker as well but same result.

Is there a way to make ADD command strip components of tar?

[cpuguy83]

That'd be nice.
Ping @tianon

[tianon]

It would be interesting to see a --strip-components on ADD, yeah, but it definitely doesn't exist today.

I think you could probably simulate it via multi-stage build, but that'll adjust all ownership to root:root when you COPY --from..., so not really a great solution for a base image like this.

ADD with a remote URL that's also a tarball to extract is something that's been discussed before and determined to be one step too far (and if the goal is to be part of the official image program, wouldn't be acceptable anyhow).

I'd really recommend re-packing your rootfs tarball to be a strict rootfs without that additional layer (even if that repacking only happens in the context of the Docker image repository).

[thaJeztah]

I think you could probably simulate it via multi-stage build, but that'll adjust all ownership to root:root when you COPY --from..., so not really a great solution for a base image like this.

I know we discussed that situation at some point, but don't recall if we created a tracking issue for that, because I think we should have an option to preserve ownership and permissions when copying between stages. I can see a lot of use-cases where a build-stage would perform all the preparations (extracting, compiling, setting permissions), and a final build-stage would only COPY --from to collect artefacts from the build-stages.

Just a bit worried of implementing everything that tar does (and trying to match the exact behaviour).

[thaJeztah]

I think you could probably simulate it via multi-stage build, but that'll adjust all ownership to root:root when you COPY --from..., so not really a great solution for a base image like this.

Small update on this part; since moby/moby#38599 (and when using buildkit, since #809), ownership should be preserved when using COPY --from between stages. Also see #790 (comment) for an example Dockerfile.

That said, there's still enhancements to be made, such as moby/moby#38710 / moby/moby#35639, and moby/moby#35280

[GrabbenD]

It's been a couple of years since this question was asked
Is there any new approach or solution to this apart from using COPY --from?

FROM scratch AS rootfs
ADD rootfs.tar.gz /

FROM scratch AS install
COPY --from=rootfs /root.x86_64 /

[thaJeztah]

I don't think there's updates on this one, but admitted, given that the alternatives with multi-stage builds aren't "too horrible", I don't think it got on the top of the priority list 😅.

Is there any new approach or solution to this apart from using COPY --from?

As you're copying to a scratch image, and won't have a shell, you could use some hacks such as described in moby/moby#38710 (comment), but I don't think those would be any cleaner than you're current approach.

Given that development of the Dockerfile syntax has fully moved to the BuildKit repository (where the definition of the syntax now lives), let me also move this ticket to that repository.