fix bug that gen layer tar contains duplicated files

[yyb196]

I run into a problem that randomly happen, image build and pushed by buildkit cannot be pulled by containerd or docker.

containerd complains that failed to convert whiteout file "etc/.wh.hosts": file exists: unknown.

docker complains that failed to register layer: Error processing tar file(exit status 1): archive/tar: invalid tar header,

or failed to register layer: Error processing tar file(duplicates of file paths not supported).

After dig a while, I found that layer which unpack fail contains duplicated files:

According to code at https://github.com/moby/buildkit/blob/master/vendor/github.com/containerd/containerd/content/local/store.go#L594 , content writer always use the already written content, this is ok for pulling content from registry, but when pushing image, content generated by snapshot comparing, in case previous cw partially write and fail, the following build will use the cached snapshot and generate layer content which append to the existing content.

In pushing case, if cw write or commit failed, we should truncate the already written file.

For guys want to know why cw write failed, there is a buildkitd log indicate the reason:

time="2023-11-20T20:14:36+08:00" level=warning msg="failed to compute blob by overlay differ (ok=false): failed to write compressed diff: mount callback failed on /tmp/containerd-mount3430795329: mount callback failed on /tmp/containerd-mount1305828223: failed to record upperdir changes (close error: failed to close tar writer: context canceled): failed to copy: context canceled"

[tonistiigi]

This is a good find. I think we want to call ContentStore.Abort() instead in the error handler. And cw.Truncate(0) after the Writer() call to make sure the writer is always empty(eg. no other client has used same ref).

[yyb196]

@tonistiigi have commit change as you suggested, PTALA.