docker load fails with "duplicates of file paths not supported"

[drigz]

If you pass a binary as data to a py_image, and this conflicts with a directory of the same name, docker load will fail with this error:

Error processing tar file(duplicates of file paths not supported):

Nothing follows the colon :(

You can get more detail as to the cause of the error by running the binary corresponding to the image:

> bazel run :main.binary
WARNING: /usr/local/google/home/rodrigoq/git/bazeltest/duplicate/BUILD.bazel:9:1:
  runfiles symlink foo/data.txt -> /usr/local/google/home/rodrigoq/git/bazeltest/duplicate/foo/data.txt
  obscured by foo -> /usr/local/google/home/rodrigoq/.cache/bazel/_bazel_rodrigoq/0ff55f53e60d60ea6f1bed429ec9f051/execroot/__main__/bazel-out/local-fastbuild/bin/foo

Then you can rename your binary to avoid the conflict.

Expected behaviour

rules_docker could print a warning or error when there are path conflicts.

Arguably, rules_docker shouldn't produce invalid tar files, although it's hard to blame it when the user has asked it to do this. I'm mostly posting this so that the error is google-able.

Repro

> cat WORKSPACE 
git_repository(
    name = "io_bazel_rules_docker",
    commit = "caa55f1e00e5909dbd689f298e2c6d3ef3e65d81",
    remote = "https://github.com/bazelbuild/rules_docker.git",
)

load(
    "@io_bazel_rules_docker//python:image.bzl",
    _py_image_repos = "repositories",
)

_py_image_repos()
> cat BUILD.bazel 
load("@io_bazel_rules_docker//python:image.bzl", "py_image")

cc_binary(
    name = "foo",
    srcs = ["foo/main.cc"],
    data = ["foo/data.txt"],
)

py_image(
    name = "main",
    srcs = ["main.py"],
    data = [":foo"],
    main = "main.py",
)
> cat foo/main.cc
int main()
{
  return 0;
}
> echo > foo/data.txt
> echo > main.py
> bazel run :main
INFO: Analysed target //:main (0 packages loaded).
INFO: Found 1 target...
Target //:main up-to-date:
  bazel-bin/main-layer.tar
INFO: Elapsed time: 1.090s, Critical Path: 0.74s
INFO: Build completed successfully, 6 total actions

INFO: Running command line: bazel-bin/main
Error response from daemon: Error processing tar file(duplicates of file paths not supported):
ERROR: Non-zero return code '1' from command: Process exited with status 1

[mattmoor]

@damienmg FYI, I think this is the same thing that blocked @ixdy's PR to pkg_tar.

I will try to reproduce this, but feel free to ping me if I let it fall through a crack as I dig myself out of Thanksgiving email hell :)

[mattmoor]

Trying to reproduce this now, I'll post back anything I find here.

[mattmoor]

Ok, there is definitely something weird going on here:

# This looks like the output of //:foo
-r-xr-xr-x 0/0            7696 1970-01-01 00:00 ./app/main.binary.runfiles/__main__/foo
# This contains the data for foo and creates the conflict.
drwxr-xr-x 0/0               0 1970-01-01 00:00 ./app/main.binary.runfiles/__main__/foo/
-r-xr-xr-x 0/0               1 1970-01-01 00:00 ./app/main.binary.runfiles/__main__/foo/data.txt

If I rename your target to something else (e.g. :binary) then things don't error...

[mattmoor]

Perhaps we're laying things out in a strange fashion, but this seems like something perhaps Bazel should reject...

[drigz]

I can see a few options:

1. Do nothing, and leave this issue as documentation.
2. Add a warning/error to build_tar (like the warning from bazel run).
3. File an upstream issue requesting that the existing warning should be detected earlier, so that users of the runfiles (like rules_docker) don't all need to detect this case themselves.

WDYT?

FYI: I found a thread on bazel-discuss where someone was confused by the shadowing behaviour (with bazel run, not rules_docker). Didn't find any other references online.

[mattmoor]

I think we at least want 2., but Bazel should probably address 3..

I'm going to assign this to @damienmg to refile this appropriately for the broader context.

[nlopezgi]

is this still an issue? please reopen if so.

[drigz]

Looks like bazel has added a warning:

WARNING: .../rules_docker_246/BUILD.bazel:9:1: runfiles symlink foo/data.txt -> foo/data.txt obscured by foo -> bazel-out/k8-fastbuild/bin/foo

So even though the latest rules_docker still has the same confusing error, Bazel's warning should help users figure it out. Seems good enough to me.