Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rewrite container copy #8570

Merged
merged 2 commits into from
Dec 4, 2020
Merged

Conversation

vrothberg
Copy link
Member

@vrothberg vrothberg commented Dec 3, 2020

  • Add a new pkg/copy to centralize all container-copy related code.

  • The new code is based on Buildah's copier package.

  • The compat /archive endpoints use the new copy package.

  • Update docs and add several new tests.

  • Includes many fixes, most notably, the look-up of volumes and mounts.

Breaking changes:

  • Podman is now expecting that container-destination paths exist.
    Before, Podman created the paths if needed. Docker does not do
    that and I believe Podman should not either as it's a recipe for
    masking errors. These errors may be user induced (e.g., a path
    typo), or internal typos (e.g., when the destination may be a
    mistakenly unmounted volume). Let's keep the magic low for such
    a security sensitive feature.

Signed-off-by: Valentin Rothberg [email protected]

@openshift-ci-robot openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 3, 2020
@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vrothberg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 3, 2020
@vrothberg
Copy link
Member Author

@edsantiago, I would love to have your feedback on the system tests. I added a lot of them. Given podman cp is very security sensitive, I thought that having a decent coverage in the e2e would is desirable.

@@ -43,7 +40,7 @@ var (
func cpFlags(cmd *cobra.Command) {
flags := cmd.Flags()
flags.BoolVar(&cpOpts.Extract, "extract", false, "Extract the tar file into the destination directory.")
flags.BoolVar(&cpOpts.Pause, "pause", copyPause(), "Pause the container while copying")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably deprecate this - it's an unnecessary addition (we fixed the CVE it was supposed to resolve in a different way).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me. Maybe something for a follow up PR when working on remote cp. Will spare us from adding the parameters.

I think we can leave it on the CLI but make it a NOP? Or full removal given 3.0 would allow us to?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave it but remove documentation, and hide the option from --help.

@vrothberg vrothberg changed the title WIP - rewrite container copy rewrite container copy Dec 3, 2020
@openshift-ci-robot openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 3, 2020
test/system/065-cp.bats Outdated Show resolved Hide resolved
test/system/065-cp.bats Outdated Show resolved Hide resolved
pkg/copy/copy.go Outdated Show resolved Hide resolved
pkg/copy/copy.go Outdated Show resolved Hide resolved
@TomSweeneyRedHat
Copy link
Member

I may have missed the convo, but any thoughts on moving copier out of Buildah and into c/common? This is one area that it would have been nice if Buildah/Podman used the same containers, we could make a bunch more of this code common between the two.

test/system/065-cp.bats Outdated Show resolved Hide resolved
I am constantly hitting the 90 seconds limit with my very slow
connection.

Signed-off-by: Valentin Rothberg <[email protected]>
@vrothberg
Copy link
Member Author

I may have missed the convo, but any thoughts on moving copier out of Buildah and into c/common? This is one area that it would have been nice if Buildah/Podman used the same containers, we could make a bunch more of this code common between the two.

SGTM

@rhatdan
Copy link
Member

rhatdan commented Dec 4, 2020

No need to block this PR. I am not crazy about moving Copier out of Buidlah since it is really tied to buildah[-copy/COPY functions in buildah.

@vrothberg vrothberg force-pushed the run-950 branch 2 times, most recently from cd5b7ec to 3096826 Compare December 4, 2020 13:28
@vrothberg
Copy link
Member Author

Changed the system tests to be table driven where applicable 👍

* Add a new `pkg/copy` to centralize all container-copy related code.

* The new code is based on Buildah's `copier` package.

* The compat `/archive` endpoints use the new `copy` package.

* Update docs and an several new tests.

* Includes many fixes, most notably, the look-up of volumes and mounts.

Breaking changes:

 * Podman is now expecting that container-destination paths exist.
   Before, Podman created the paths if needed.  Docker does not do
   that and I believe Podman should not either as it's a recipe for
   masking errors.  These errors may be user induced (e.g., a path
   typo), or internal typos (e.g., when the destination may be a
   mistakenly unmounted volume).  Let's keep the magic low for such
   a security sensitive feature.

Signed-off-by: Valentin Rothberg <[email protected]>
@vrothberg
Copy link
Member Author

@containers/podman-maintainers PTAL

@TomSweeneyRedHat
Copy link
Member

Looks like the ' rdoproject.org/github-check' is failing here and in other PRs this morning with a retry limit failure of F31. podman-package-fedora-31.

I'm not sure if there's anything we can do with that.

@TomSweeneyRedHat
Copy link
Member

LGTM
FWIW, I was not suggesting moving copier pkg to common as part of this work, just something to consider for the future.

@rhatdan
Copy link
Member

rhatdan commented Dec 4, 2020

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Dec 4, 2020
@openshift-merge-robot openshift-merge-robot merged commit 0f496e4 into containers:master Dec 4, 2020
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 24, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants