Replacing a directory with a symlink seems to confuse the scripts #1189

raphaelcoeffic · 2022-10-28T12:05:56Z

Describe the bug

When running tern on a docker image that has this statement:

RUN \
    echo "\n#\n#\n# creating symlink from old root dir\n#\n#\n" && \
    rm -rf /root && ln -s /data/root /root

The following error is spilled while tern is processing the layer:

2022-10-27 15:43:07,918 - DEBUG - common - Reading files in filesystem...
2022-10-27 15:43:07,919 - DEBUG - rootfs - Running command: cp -r /root/.tern/temp/10/contents/root /root/.tern/temp/mergedir
2022-10-27 15:43:08,186 - ERROR - rootfs - Command failed. cp: cannot overwrite directory '/root/.tern/temp/mergedir/root' with non-directory

Traceback (most recent call last):
  File "/usr/local/bin/tern", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/tern/__main__.py", line 309, in main
    do_main(args)
  File "/usr/local/lib/python3.9/site-packages/tern/__main__.py", line 123, in do_main
    crun.execute_image(args)
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/run.py", line 80, in execute_image
    cimage.default_analyze(full_image, args)
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/image.py", line 74, in default_analyze
    multi_layer.analyze_subsequent_layers(
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/multi_layer.py", line 168, in analyze_subsequent_layers
    fresh_analysis(image_obj, curr_layer, prereqs, options)
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/multi_layer.py", line 113, in fresh_analysis
    target = prep_layers(image_obj, curr_layer, options.driver)
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/multi_layer.py", line 71, in prep_layers
    return apply_layers(image_obj, top_layer)
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/multi_layer.py", line 63, in apply_layers
    rootfs.root_command(['cp', '-r'] + glob.glob(layer_contents), target)
  File "/usr/local/lib/python3.9/site-packages/tern/utils/rootfs.py", line 71, in root_command
    raise subprocess.CalledProcessError(  # nosec
subprocess.CalledProcessError: Command '['cp', '-r', '/root/.tern/temp/10/contents/root', '/root/.tern/temp/mergedir']' returned non-zero exit status 1.

To Reproduce
Steps to reproduce the behavior:

Make a Dockerfile that replaces /root directory with a symlink somewhere else

ARG version=11
ARG img=debian

FROM ${img}:${version}

RUN mkdir -p /data

RUN \
    echo "\n#\n#\n# creating symlink from old root dir\n#\n#\n" && \
    rm -rf /root && ln -s /data/root /root

Build the container image

docker build -t registry.mycompany.com/debian-systemd-tern-test .

Push to registry

docker push registry.mycompany.com/debian-systemd-tern-test

Run tern on that container image

❯ docker run -it --rm -v /etc/docker/certs.d:/etc/containers/certs.d ternd report -i registry.mycompany.com/debian-systemd-tern-test:latest                               tern-test 14:10:28
2022-10-28 12:10:35,892 - DEBUG - __main__ - Starting...
2022-10-28 12:10:35,892 - DEBUG - prep - Setting up...
2022-10-28 12:10:35,893 - DEBUG - rootfs - Running command: chmod +x /usr/local/lib/python3.9/site-packages/tern/tools/fs_hash.sh
2022-10-28 12:10:36,072 - DEBUG - run - Starting analysis...
2022-10-28 12:10:36,073 - DEBUG - skopeo - Attempting to pull image "registry.mycompany.com/debian-systemd-tern-test:latest"
2022-10-28 12:10:36,073 - DEBUG - rootfs - Running command: skopeo copy docker://registry.mycompany.com/debian-systemd-tern-test:latest dir:/root/.tern/temp
2022-10-28 12:10:40,253 - DEBUG - skopeo - Inspecting remote image "registry.mycompany.com/debian-systemd-tern-test:latest"
2022-10-28 12:10:40,254 - DEBUG - rootfs - Running command: skopeo inspect docker://registry.mycompany.com/debian-systemd-tern-test:latest
2022-10-28 12:10:40,434 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/17c9e6141fdb3387e5a1c07d4f9b6a05ac1498e96029fa3ea55470d4504f7770
2022-10-28 12:10:41,393 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/17c9e6141fdb3387e5a1c07d4f9b6a05ac1498e96029fa3ea55470d4504f7770 -C /root/.tern/temp/1/contents
2022-10-28 12:10:43,303 - DEBUG - rootfs - Running command: /usr/local/lib/python3.9/site-packages/tern/tools/fs_hash.sh /root/.tern/temp/1/contents
2022-10-28 12:11:02,121 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/67649db123fc6683600860faa84abd4735d3539d70ec4a65451d94ea357f38a3
2022-10-28 12:11:02,182 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/67649db123fc6683600860faa84abd4735d3539d70ec4a65451d94ea357f38a3 -C /root/.tern/temp/2/contents
2022-10-28 12:11:02,192 - DEBUG - rootfs - Running command: /usr/local/lib/python3.9/site-packages/tern/tools/fs_hash.sh /root/.tern/temp/2/contents
2022-10-28 12:11:02,207 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/de5f46f28e027af45576d5155b3e29d574a5153de48726416de2c6bae8934100
2022-10-28 12:11:02,235 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/de5f46f28e027af45576d5155b3e29d574a5153de48726416de2c6bae8934100 -C /root/.tern/temp/3/contents
2022-10-28 12:11:02,242 - DEBUG - rootfs - Running command: /usr/local/lib/python3.9/site-packages/tern/tools/fs_hash.sh /root/.tern/temp/3/contents
2022-10-28 12:11:02,250 - DEBUG - common - Reading files in filesystem...
2022-10-28 12:11:05,065 - DEBUG - rootfs - Running command: tar -tf /root/.tern/temp/17c9e6141fdb3387e5a1c07d4f9b6a05ac1498e96029fa3ea55470d4504f7770
2022-10-28 12:11:06,011 - DEBUG - rootfs - Running command: tar -x -f /root/.tern/temp/17c9e6141fdb3387e5a1c07d4f9b6a05ac1498e96029fa3ea55470d4504f7770 -C /root/.tern/temp/mergedir
2022-10-28 12:11:09,984 - DEBUG - rootfs - Running command: mknod /root/.tern/temp/mergedir/dev/urandom c 1 9
2022-10-28 12:11:10,022 - DEBUG - rootfs - Running command: cp /etc/resolv.conf /root/.tern/temp/mergedir/etc/resolv.conf
2022-10-28 12:11:10,038 - DEBUG - core - Collecting metadata for image layer...
2022-10-28 12:11:10,156 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c dpkg-query -W -f '${Version}
'
2022-10-28 12:11:10,319 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c pkgs=`dpkg-query -W -f '${Package}
'` && for p in $pkgs; do dpkg-query -f '${source:Version}
' -W $p; done
2022-10-28 12:11:10,580 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c pkgs=`dpkg-query -W -f '${Package}
'` && for p in $pkgs; do dpkg-query -f '${source:Package}
' -W $p; done
2022-10-28 12:11:10,831 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c pkgs=`dpkg-query -W -f '${Package}
'` && for p in $pkgs; do files=`dpkg-query -L $p`; for file in $files; do if [ -f $file ]; then echo $file; fi; done; echo LICF; done
2022-10-28 12:11:11,370 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c pkgs=`dpkg-query -W -f '${Package}
'` && for p in $pkgs; do /bin/cat /usr/share/doc/$p/copyright; echo LICF; done
2022-10-28 12:11:11,489 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c dpkg-query -W -f '${Package}
'
2022-10-28 12:11:11,496 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c dpkg-query -W -f '${Homepage}
' | /bin/sed 's/^$/Unknown/'
2022-10-28 12:11:11,523 - DEBUG - core - Processing Debian copyrights...
2022-10-28 12:11:11,731 - WARNING - core - Some metadata may be missing
2022-10-28 12:11:23,668 - DEBUG - common - Reading files in filesystem...
2022-10-28 12:11:23,686 - DEBUG - rootfs - Running command: cp -r /root/.tern/temp/2/contents/data /root/.tern/temp/mergedir
2022-10-28 12:11:23,885 - WARNING - command_lib - No listing method for 'mkdir'. Additional analysis may be required.

2022-10-28 12:11:23,886 - DEBUG - core - Collecting metadata for image layer...
2022-10-28 12:11:23,886 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" && dpkg-query -W -f '${Version}
'
2022-10-28 12:11:23,900 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" && pkgs=`dpkg-query -W -f '${Package}
'` && for p in $pkgs; do dpkg-query -f '${source:Version}
' -W $p; done
2022-10-28 12:11:24,308 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" && pkgs=`dpkg-query -W -f '${Package}
'` && for p in $pkgs; do dpkg-query -f '${source:Package}
' -W $p; done
2022-10-28 12:11:24,580 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" && pkgs=`dpkg-query -W -f '${Package}
'` && for p in $pkgs; do files=`dpkg-query -L $p`; for file in $files; do if [ -f $file ]; then echo $file; fi; done; echo LICF; done
2022-10-28 12:11:24,913 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" && pkgs=`dpkg-query -W -f '${Package}
'` && for p in $pkgs; do /bin/cat /usr/share/doc/$p/copyright; echo LICF; done
2022-10-28 12:11:25,009 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" && dpkg-query -W -f '${Package}
'
2022-10-28 12:11:25,016 - DEBUG - rootfs - Running command: chroot /root/.tern/temp/mergedir /bin/sh -c export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" && dpkg-query -W -f '${Homepage}
' | /bin/sed 's/^$/Unknown/'
2022-10-28 12:11:25,024 - DEBUG - core - Processing Debian copyrights...
2022-10-28 12:11:25,218 - WARNING - core - Some metadata may be missing
2022-10-28 12:11:25,445 - DEBUG - rootfs - Running command: cp -r /root/.tern/temp/2/contents/data /root/.tern/temp/mergedir
2022-10-28 12:11:32,523 - DEBUG - common - Reading files in filesystem...
2022-10-28 12:11:32,523 - DEBUG - rootfs - Running command: cp -r /root/.tern/temp/3/contents/root /root/.tern/temp/mergedir
2022-10-28 12:11:32,527 - ERROR - rootfs - Command failed. cp: cannot overwrite directory '/root/.tern/temp/mergedir/root' with non-directory

Traceback (most recent call last):
  File "/usr/local/bin/tern", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/tern/__main__.py", line 309, in main
    do_main(args)
  File "/usr/local/lib/python3.9/site-packages/tern/__main__.py", line 123, in do_main
    crun.execute_image(args)
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/run.py", line 80, in execute_image
    cimage.default_analyze(full_image, args)
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/image.py", line 74, in default_analyze
    multi_layer.analyze_subsequent_layers(
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/multi_layer.py", line 168, in analyze_subsequent_layers
    fresh_analysis(image_obj, curr_layer, prereqs, options)
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/multi_layer.py", line 113, in fresh_analysis
    target = prep_layers(image_obj, curr_layer, options.driver)
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/multi_layer.py", line 71, in prep_layers
    return apply_layers(image_obj, top_layer)
  File "/usr/local/lib/python3.9/site-packages/tern/analyze/default/container/multi_layer.py", line 63, in apply_layers
    rootfs.root_command(['cp', '-r'] + glob.glob(layer_contents), target)
  File "/usr/local/lib/python3.9/site-packages/tern/utils/rootfs.py", line 71, in root_command
    raise subprocess.CalledProcessError(  # nosec
subprocess.CalledProcessError: Command '['cp', '-r', '/root/.tern/temp/3/contents/root', '/root/.tern/temp/mergedir']' returned non-zero exit status 1.

Environment you are running Tern on

Tern is run within a Docker container.

Output of 'tern --version'

Version: 2.10.1

Operating System (Linux Distro and version or Mac or Windows)

❯ cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)

❯ uname -a
Linux docker 3.10.0-1160.76.1.el7.x86_64 #1 SMP Wed Aug 10 16:21:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Python version (3.6 or higher):

❯ docker run -it --rm --entrypoint /bin/bash ternd
root@7d03c9e9c32b:/# python --version
Python 3.9.15

The text was updated successfully, but these errors were encountered:

rnjudge · 2022-11-29T20:54:48Z

Thanks for this issue @raphaelcoeffic -- I'll take a look!

rnjudge · 2022-12-13T00:04:34Z

The issue is with the bulk copying of the layer contents using cp -r when dealing with symlinks at the directory level. cp doesn't recognize the unzipped directory symlink, /root/.tern/temp/3/contents/root (symlinked to -> /data/root) to be a valid directory replacement for copying to the working directory. Looping in @nishakm for ideas.

Adding the -L option to cp and running cp -rL, which may properly copy the adds significant overhead to the runtime (20+ minutes before I cancelled it vs 15 seconds for cp -r). rsync may be an option?

nishakm · 2023-01-03T19:56:56Z

@rnjudge To clarify, does the error occur when running cp -r container_layer_dir working_dir where container_layer_dir is a symlink to random_dir_path?

rnjudge · 2023-01-03T21:25:38Z

@rnjudge To clarify, does the error occur when running cp -r container_layer_dir working_dir where container_layer_dir is a symlink to random_dir_path?

@nishakm Yep! In this case, the dockerfile has something like: ln -s /data/root /root in it and during analysis Tern tries to cp -r /root/.tern/temp/3/contents/root /root/.tern/temp/mergedir but /root/.tern/temp/3/contents/root is a symlink and therefore doesn't get recognized as a directory that can be copied to /root/.tern/temp/mergedir.

raphaelcoeffic changed the title ~~Replace a directory with a symlink seems to confuse the scripts~~ Replacing a directory with a symlink seems to confuse the scripts Oct 28, 2022

rnjudge mentioned this issue Jun 28, 2023

Problem with broken symlinks while scaning some containers #1234

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replacing a directory with a symlink seems to confuse the scripts #1189

Replacing a directory with a symlink seems to confuse the scripts #1189

raphaelcoeffic commented Oct 28, 2022 •

edited

Loading

rnjudge commented Nov 29, 2022

rnjudge commented Dec 13, 2022

nishakm commented Jan 3, 2023

rnjudge commented Jan 3, 2023

Replacing a directory with a symlink seems to confuse the scripts #1189

Replacing a directory with a symlink seems to confuse the scripts #1189

Comments

raphaelcoeffic commented Oct 28, 2022 • edited Loading

rnjudge commented Nov 29, 2022

rnjudge commented Dec 13, 2022

nishakm commented Jan 3, 2023

rnjudge commented Jan 3, 2023

raphaelcoeffic commented Oct 28, 2022 •

edited

Loading