bugtool: Compress and copy with resume #671
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit is to add gzip compression for cilium-bugtool to reduce size of transferred file. In local cluster, the size is reduced significantly (e.g. from 21 MB to ~3MB). Once copied to local system, this file is extracted again so that there is no change in existing behavior. ``` root@kind-control-plane:/tmp# ls -lrth cilium-bugtool-20211224-072* -rw-r--r-- 1 root root 21M Dec 24 07:20 cilium-bugtool-20211224-072047.092+0000-UTC-1266452003.tar -rw-r--r-- 1 root root 3.1M Dec 24 07:21 cilium-bugtool-20211224-072122.815+0000-UTC-2647778632.tar.gz ``` Relates to cilium#616 (comment) Signed-off-by: Tam Mach <[email protected]>
sayboras
force-pushed
the
tam/transfer
branch
from
9db18bd to
a81b815
Compare
This commit is to add CopyFromPod function in current k8s client, which supports resuming copy from last byte offset. Small refactor for exec in pod method was done to avoid copying reader multiple times. Relates to cilium#616 (comment) Signed-off-by: Tam Mach <[email protected]>
sayboras
force-pushed
the
tam/transfer
branch
from
a81b815 to
a0aa326
Compare
tklauser
approved these changes
Jan 3, 2022
twpayne
reviewed
Jan 4, 2022
| @@ -0,0 +1,46 @@ | |||
| // SPDX-License-Identifier: Apache-2.0 | |||
| // Copyright 2020 Authors of Cilium | |||
There was a problem hiding this comment.
nit: The copyright year should be 2021. This also applies to other files.
| outFile, err := os.Create(destFile) | ||
| if err != nil { | ||
| return err | ||
| } |
There was a problem hiding this comment.
This function is missing a defer outFile.Close() or similar.
There was a problem hiding this comment.
you are right, I forgot about this most of the time, and relying on linters. If I am not wrong, file is having finalizer i.e. fd will be garbage collected later, not idea in case of reuse of file descriptor.
Let me make a small update on this, thanks again.
sayboras
added a commit
to sayboras/cilium-cli
that referenced
this pull request
Jan 4, 2022
This commit is to correct/add header with year 2022 for new files. Relates cilium#671 (comment)
sayboras
added a commit
to sayboras/cilium-cli
that referenced
this pull request
Jan 4, 2022
This commit is to correct/add header with year 2022 for new files. Relates cilium#671 (comment) Signed-off-by: Tam Mach <[email protected]>
christarazi
pushed a commit
that referenced
this pull request
Jan 4, 2022
This commit is to correct/add header with year 2022 for new files. Relates #671 (comment) Signed-off-by: Tam Mach <[email protected]>
2 tasks
aditighag
pushed a commit
to aditighag/cilium-cli
that referenced
this pull request
Apr 21, 2023
This commit is to correct/add header with year 2022 for new files. Relates cilium#671 (comment) Signed-off-by: Tam Mach <[email protected]>
michi-covalent
pushed a commit
to michi-covalent/cilium
that referenced
this pull request
May 30, 2023
This commit is to correct/add header with year 2022 for new files. Relates cilium/cilium-cli#671 (comment) Signed-off-by: Tam Mach <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR tries to address the below points mentioned in #616
Changes
Please refer to individual commit for more details
Testing
Testing was done as per below
Make sure that the existing functionalities still working
Testing with a very large file (~10GB before compressed, 50MB after compressed)