Using OCI images instead of Docker Images #1006

mukultaneja · 2021-07-15T19:52:51Z

Start analyzing OCI images using tern

Work towards: #948

Signed-off-by: Mukul Taneja [email protected]

mukultaneja · 2021-07-15T19:55:09Z

@nishakm @rnjudge, Initial commit towards Issue #948. I would keep updating the same PR with small changes and request you to review them step by step.

nishakm

Sorry for reviewing this so late! Thanks for continuing to work on this! It looks good to me. I wonder if you can add some tests for the OCIImage class?
Also, there are some prospector errors that come up, but seem to be easy to resolve. Let me know if you're stuck with anything.

tern/analyze/default/container/image.py

tern/analyze/oci/parse.py

mukultaneja · 2021-08-31T20:34:16Z

@nishakm @rnjudge, Please review this PR. I have made the required changes and tested with the below command
tern -c report -i docker://debian:latest. This PR does the following things,

Takes docker image as input and convert into OCI format using skopeo utility internally
Contains respective class to load OCI image
Follows the same image analysis process after loading the image.

nishakm

@mukultaneja Sorry for taking so long to get back to you on this. I have one comment on the implementation as it is causing a problem with the security linter. Also, can you rebase your change so you can pick up the latest prospector changes?

tern/analyze/default/container/image.py

mukultaneja · 2021-09-23T20:47:18Z

@nishakm, I tried to get away with security issue by having shell==True / False but seems the linter does not allow to use subprocess at all. Any suggestions?

nishakm

@mukultaneja I've added a suggestion. I will also try to test the changes.

tern/analyze/default/container/image.py

tern/classes/oci_image.py

Start analyzing OCI images using tern Work towards: #948 Signed-off-by: Mukul Taneja <[email protected]>

Adding 'download_container_image` method Work towards: #948 Signed-off-by: mtaneja <[email protected]>

Updating OCI Image class with required methods to load image. Work towards: #948 Signed-off-by: Mukul Taneja <[email protected]>

mukultaneja · 2021-09-29T21:31:09Z

@nishakm I made all the requested changes and this time all tests got passed too :)

nishakm

@rnjudge Need your input on this PR :)

nishakm · 2021-10-06T15:41:59Z

tern/analyze/default/container/image.py

+        image_attr = general.parse_image_string(docker_image)
+        oci_image = 'oci://{0}/{1}'.format(
+            rootfs.working_dir, image_attr.get('name'))
+        docker_image = 'docker://{0}'.format(docker_image)


I just found out that if we use docker:// rather than dockerdaemon we encounter a TLS error on the registry side. I also found that crane is a much more lightweight tool to pull container images, and conforms with OCI's on-disk layout spec. @rnjudge Do you want to merge this PR and then let @mukultaneja update it to use crane?

What is the effect of the TLS error on the registry side?

You can't download the image, because the certificate has expired or something like that. Usually, you need to set an option to disable TLS checking on the client side.

The disadvantage of switching to crane is that you have to download the binary from github rather than install it via the package manager. But I feel that the install is easy and the source code is included with the binary distribution.

nishakm · 2021-11-30T15:18:39Z

Hi @mukultaneja, sorry again for the delay in reviewing! I tried running tern with these changes. I get the following error:

2021-11-30 07:13:17,531 - DEBUG - rootfs - Running command: skopeo copy docker://debian:buster oci:///home/nisha/.tern/debian
Traceback (most recent call last):
  File "/home/nisha/terndev/bin/tern", line 10, in <module>
    sys.exit(main())
  File "/home/nisha/terndev/tern/tern/__main__.py", line 287, in main
    do_main(args)
  File "/home/nisha/terndev/tern/tern/__main__.py", line 116, in do_main
    crun.execute_image(args)
  File "/home/nisha/terndev/tern/tern/analyze/default/container/run.py", line 65, in execute_image
    full_image = cimage.load_full_image(
  File "/home/nisha/terndev/tern/tern/analyze/default/container/image.py", line 54, in load_full_image
    test_image.load_image(load_until_layer)
  File "/home/nisha/terndev/tern/tern/classes/oci_image.py", line 130, in load_image
    self._image_id = self.get_image_id(self._manifest)
AttributeError: 'OCIImage' object has no attribute 'get_image_id'

Also, could you please rebase your changes on top of main?

nishakm · 2021-11-30T16:32:52Z

@mukultaneja few other requests:

Could you give more descriptive titles for each of your commits? I see each one having: "Using OCI images instead of Docker Images"
Could you add a commit with documentation changes to https://github.com/tern-tools/tern#getting-started-on-linux and changes to the various dockerfiles and the Vagrantfile to install skopeo?

nishakm · 2021-12-09T22:46:29Z

Hi @mukultaneja. I have included your changes in #1090. Please take a look. I will close this PR once the more updated one is merged. Thanks again for your work on this!

nishakm requested changes Aug 5, 2021

View reviewed changes

tern/analyze/default/container/image.py Outdated Show resolved Hide resolved

tern/analyze/oci/parse.py Outdated Show resolved Hide resolved

nishakm requested changes Sep 14, 2021

View reviewed changes

tern/analyze/default/container/image.py Outdated Show resolved Hide resolved

mukultaneja requested a review from nishakm September 23, 2021 20:38

nishakm requested changes Sep 23, 2021

View reviewed changes

tern/analyze/default/container/image.py Outdated Show resolved Hide resolved

tern/analyze/default/container/image.py Outdated Show resolved Hide resolved

tern/classes/oci_image.py Outdated Show resolved Hide resolved

mukultaneja and others added 3 commits September 30, 2021 02:48

Using OCI images instead of Docker Images

e3bdc51

Start analyzing OCI images using tern Work towards: #948 Signed-off-by: Mukul Taneja <[email protected]>

Using OCI images instead of Docker Images

65705b6

Adding 'download_container_image` method Work towards: #948 Signed-off-by: mtaneja <[email protected]>

Using OCI images instead of Docker Images

7084715

Updating OCI Image class with required methods to load image. Work towards: #948 Signed-off-by: Mukul Taneja <[email protected]>

mukultaneja requested a review from nishakm September 29, 2021 21:30

nishakm reviewed Oct 6, 2021

View reviewed changes

nishakm closed this Dec 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Using OCI images instead of Docker Images #1006

Using OCI images instead of Docker Images #1006

mukultaneja commented Jul 15, 2021

mukultaneja commented Jul 15, 2021

nishakm left a comment

mukultaneja commented Aug 31, 2021 •

edited

Loading

nishakm left a comment

mukultaneja commented Sep 23, 2021

nishakm left a comment

mukultaneja commented Sep 29, 2021

nishakm left a comment

nishakm Oct 6, 2021

rnjudge Oct 6, 2021

nishakm Oct 7, 2021

nishakm Oct 7, 2021

nishakm commented Nov 30, 2021

nishakm commented Nov 30, 2021

nishakm commented Dec 9, 2021

Using OCI images instead of Docker Images #1006

Using OCI images instead of Docker Images #1006

Conversation

mukultaneja commented Jul 15, 2021

mukultaneja commented Jul 15, 2021

nishakm left a comment

Choose a reason for hiding this comment

mukultaneja commented Aug 31, 2021 • edited Loading

nishakm left a comment

Choose a reason for hiding this comment

mukultaneja commented Sep 23, 2021

nishakm left a comment

Choose a reason for hiding this comment

mukultaneja commented Sep 29, 2021

nishakm left a comment

Choose a reason for hiding this comment

nishakm Oct 6, 2021

Choose a reason for hiding this comment

rnjudge Oct 6, 2021

Choose a reason for hiding this comment

nishakm Oct 7, 2021

Choose a reason for hiding this comment

nishakm Oct 7, 2021

Choose a reason for hiding this comment

nishakm commented Nov 30, 2021

nishakm commented Nov 30, 2021

nishakm commented Dec 9, 2021

mukultaneja commented Aug 31, 2021 •

edited

Loading