Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support progress output in pull #1148

Merged
merged 139 commits into from
Oct 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
139 commits
Select commit Hold shift + click to select a range
b088f2b
feat: support progressed output in blob push and blob get
qweeah Sep 12, 2023
e773471
add error when --debug is used with terminal output
qweeah Sep 19, 2023
6044594
bug fix
qweeah Sep 19, 2023
b5d89dd
Merge branch 'main' into progress-bar-reader
qweeah Sep 20, 2023
6324b93
add test for console
qweeah Sep 20, 2023
f1077bd
fix lint & pass tty to manager
qweeah Sep 20, 2023
4c362f8
Merge branch 'main' into progress-bar-reader
qweeah Sep 20, 2023
941188f
rename flag to disable tty
qweeah Sep 20, 2023
c72ebe5
add test for blob push
qweeah Sep 21, 2023
5fc75a8
fix render racing
qweeah Sep 21, 2023
a2d3f8c
fix race
qweeah Sep 21, 2023
4a065fe
add test for `blob fetch`
qweeah Sep 21, 2023
00ca2fc
resolve comment
qweeah Sep 22, 2023
c85e809
resolve comment
qweeah Sep 22, 2023
997a63e
resolve comment
qweeah Sep 22, 2023
144e907
fix lint
qweeah Sep 22, 2023
1b1f23c
refactor spinner mark
qweeah Sep 22, 2023
008a61d
refactor: improve manager and status
qweeah Sep 25, 2023
95ad473
add error for manager functions
qweeah Sep 25, 2023
ed84a69
remove unnecessary lock
qweeah Sep 25, 2023
47ef494
update reader naming
qweeah Sep 25, 2023
7e249ed
remove reader.once
qweeah Sep 25, 2023
ca19602
Merge remote-tracking branch 'origin_src/main' into progress-bar-reader
qweeah Sep 25, 2023
a6476d4
ignore returned error of manager
qweeah Sep 25, 2023
8c7458b
code clean
qweeah Sep 25, 2023
1584fd2
resolve comment
qweeah Sep 28, 2023
8fef80b
Merge remote-tracking branch 'origin_src/main' into progress-bar-reader
qweeah Sep 28, 2023
b44e2e8
code clean
qweeah Sep 28, 2023
ea6476f
code clean
qweeah Sep 28, 2023
b06383c
feat: support progress bar for `oras cp`
qweeah Sep 28, 2023
720c6e6
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Sep 28, 2023
ebcc114
Merge remote-tracking branch 'origin_src/main' into progress-bar-reader
qweeah Oct 7, 2023
ad0608d
add test for spinner
qweeah Oct 7, 2023
1aab8bd
add cross-package coverage generation
qweeah Oct 7, 2023
78b0d42
Refactor tests
qweeah Oct 7, 2023
2b13d6a
cover common tests
qweeah Oct 7, 2023
6bf4f3d
add status coverage
qweeah Oct 7, 2023
a7f3f96
code clean
qweeah Oct 7, 2023
cfc7a9b
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 7, 2023
5183dc9
add doc for stdout
qweeah Oct 7, 2023
1f45188
add status tests
qweeah Oct 7, 2023
7133edd
resolve comment
qweeah Oct 7, 2023
3ad8eaa
add doc for stdout
qweeah Oct 7, 2023
243dc37
add status tests
qweeah Oct 7, 2023
5f5a1d9
resolve comment
qweeah Oct 7, 2023
7554534
NIT
qweeah Oct 7, 2023
76de650
align timing and percentage
qweeah Oct 7, 2023
51ab043
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 7, 2023
486a8ed
fix unit test
qweeah Oct 7, 2023
5b24511
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 7, 2023
39b0247
Merge remote-tracking branch 'origin_src/main' into progress-bar-reader
qweeah Oct 7, 2023
4fe990e
increase coverage
qweeah Oct 7, 2023
3a7ed21
fix: won't show complete when aborted
qweeah Oct 8, 2023
e916c17
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 8, 2023
455ca08
fix: won't show complete when aborted
qweeah Oct 8, 2023
ff2c230
code clean
qweeah Oct 8, 2023
9426fac
code clean
qweeah Oct 8, 2023
50c4181
code clean
qweeah Oct 8, 2023
aa5b847
code clean
qweeah Oct 8, 2023
f49bdcb
align units of processed data to total size
qweeah Oct 8, 2023
1b774e0
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 8, 2023
8bc5c37
add instant download speed
qweeah Oct 8, 2023
5ed9251
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 8, 2023
e1486cc
remove unnecessary file
qweeah Oct 8, 2023
3193b48
code clean
qweeah Oct 8, 2023
a46c218
code clean
qweeah Oct 8, 2023
151ad0a
lint
qweeah Oct 8, 2023
7327022
code clean
qweeah Oct 9, 2023
1abda95
change base for sizing
qweeah Oct 9, 2023
636f424
fix speed display
qweeah Oct 9, 2023
2ea395d
add test coverage
qweeah Oct 9, 2023
d7276cb
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 9, 2023
feb73b4
add test for copying
qweeah Oct 9, 2023
4f7ac01
bug fix
qweeah Oct 9, 2023
25c155a
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 9, 2023
cfdfe46
code clean
qweeah Oct 9, 2023
8c71709
feat: support progress output in attach and push
qweeah Oct 9, 2023
b9c5bec
fix nil pointer
qweeah Oct 9, 2023
c47f967
reset render interval to 200ms
qweeah Oct 9, 2023
2489a61
turn humanize into package
qweeah Oct 10, 2023
9e7daec
bug fix
qweeah Oct 10, 2023
c4564fe
rename pts device
qweeah Oct 10, 2023
167adb1
Merge remote-tracking branch 'origin_src/main' into progress-bar-reader
qweeah Oct 10, 2023
6c324d1
reduce buffer size
qweeah Oct 10, 2023
d4298c1
fix typo
qweeah Oct 10, 2023
67574dd
change switch to if-else
qweeah Oct 10, 2023
25d8935
bug fix
qweeah Oct 10, 2023
c76c464
change switch to if-else
qweeah Oct 10, 2023
9403cca
revert rendering
qweeah Oct 10, 2023
8a1f088
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 10, 2023
0741396
reduce buffer size to 1
qweeah Oct 10, 2023
2439121
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 11, 2023
85d8514
Merge branch 'progress-bar-cp' into progress-bar-push-attach
qweeah Oct 11, 2023
8d6c017
fix: enable pipe when output to stdout
qweeah Oct 11, 2023
c5ab1ee
fix: enable pipe when output to stdout
qweeah Oct 11, 2023
360a5ea
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 11, 2023
6611186
fix display bug
qweeah Oct 11, 2023
c4fae1f
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 11, 2023
e858d1a
revert change
qweeah Oct 11, 2023
8692963
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 11, 2023
49818cf
Merge branch 'progress-bar-cp' into progress-bar-push-attach
qweeah Oct 11, 2023
e4d4439
fix bug
qweeah Oct 11, 2023
6f2de09
Merge branch 'progress-bar-reader' into progress-bar-cp
qweeah Oct 11, 2023
b01433d
Merge branch 'progress-bar-cp' into progress-bar-push-attach
qweeah Oct 11, 2023
7c7510f
add skipped status
qweeah Oct 13, 2023
e26cf77
Merge remote-tracking branch 'origin_src/main' into progress-bar-push…
qweeah Oct 13, 2023
eba5761
fix merging
qweeah Oct 13, 2023
6a97757
fix pushing
qweeah Oct 13, 2023
1a4c490
fix cp
qweeah Oct 13, 2023
a4b0379
Merge branch 'main' into progress-bar-push-attach
qweeah Oct 16, 2023
781874f
resolve comment
qweeah Oct 17, 2023
0dd031a
change target to graph target
qweeah Oct 17, 2023
d2c8f16
resolve comment
qweeah Oct 17, 2023
0de9997
remove printStatus output target.Prompt
qweeah Oct 17, 2023
e0f810a
fix attach
qweeah Oct 17, 2023
a481b48
feat: support progress bar for `oras pull`
qweeah Oct 17, 2023
5e33060
refactor target
qweeah Oct 17, 2023
3fd346e
apply to pull
qweeah Oct 17, 2023
060940b
refactor target
qweeah Oct 17, 2023
692fd31
resolve comment
qweeah Oct 17, 2023
8a52cc9
doc clean
qweeah Oct 17, 2023
1ccb8d1
Merge remote-tracking branch 'origin_src/main' into progress-bar-push…
qweeah Oct 18, 2023
33feeb8
Merge branch 'progress-bar-push-attach' into progress-pull
qweeah Oct 18, 2023
a3d9e68
complete merge
qweeah Oct 18, 2023
adc689f
Merge branch 'main' into progress-pull
qweeah Oct 23, 2023
4fee3d8
revert push changes
qweeah Oct 23, 2023
d45f8b1
code clean
qweeah Oct 23, 2023
266f5c9
align comment
qweeah Oct 23, 2023
3ea07c2
add target test
qweeah Oct 23, 2023
581d03f
add test & code clean
qweeah Oct 23, 2023
add2ac0
fix UT and linter
qweeah Oct 23, 2023
bd98c93
fix lint
qweeah Oct 23, 2023
d72c4c6
fix typo
qweeah Oct 23, 2023
ec04777
fix UT
qweeah Oct 23, 2023
bbeb48e
resolve comment
qweeah Oct 27, 2023
c4f161d
resolve comment
qweeah Oct 27, 2023
44a0de7
remove extra logs
qweeah Oct 27, 2023
b8bf1e3
resolve comment
qweeah Oct 27, 2023
0133330
Merge remote-tracking branch 'origin_src/main' into progress-pull
qweeah Oct 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/oras/internal/display/progress/manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ const (
bufFlushDuration = 200 * time.Millisecond
)

var errManagerStopped = errors.New("progress output manage has already been stopped")
var errManagerStopped = errors.New("progress output manager has already been stopped")

// Status is print message channel
type Status chan *status
Expand Down
4 changes: 2 additions & 2 deletions cmd/oras/internal/display/track/target.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ import (
type GraphTarget interface {
oras.GraphTarget
io.Closer
Prompt(desc ocispec.Descriptor, prompt string, verbose bool) error
Prompt(desc ocispec.Descriptor, prompt string) error
}

type graphTarget struct {
Expand Down Expand Up @@ -102,7 +102,7 @@ func (t *graphTarget) Close() error {
}

// Prompt prompts the user with the provided prompt and descriptor.
func (t *graphTarget) Prompt(desc ocispec.Descriptor, prompt string, verbose bool) error {
func (t *graphTarget) Prompt(desc ocispec.Descriptor, prompt string) error {
status, err := t.manager.Add()
if err != nil {
return err
Expand Down
112 changes: 112 additions & 0 deletions cmd/oras/internal/display/track/target_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
/*
Copyright The ORAS Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package track

import (
"bytes"
"context"
"io"
"testing"

"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"oras.land/oras-go/v2"
"oras.land/oras-go/v2/content/memory"
"oras.land/oras/cmd/oras/internal/display/console/testutils"
)

type testReferenceGraphTarget struct {
oras.GraphTarget
}

func (t *testReferenceGraphTarget) PushReference(ctx context.Context, expected ocispec.Descriptor, content io.Reader, reference string) error {
err := t.GraphTarget.Push(ctx, expected, content)
if err != nil {
return err
}
return t.GraphTarget.Tag(ctx, expected, reference)
}

func Test_referenceGraphTarget_PushReference(t *testing.T) {
// prepare
pty, device, err := testutils.NewPty()
if err != nil {
t.Fatal(err)
}
defer device.Close()
src := memory.New()
content := []byte("test")
r := bytes.NewReader(content)
desc := ocispec.Descriptor{
MediaType: "application/octet-stream",
Digest: digest.FromBytes(content),
Size: int64(len(content)),
}
// test
tag := "tagged"
actionPrompt := "action"
donePrompt := "done"
target, err := NewTarget(&testReferenceGraphTarget{src}, actionPrompt, donePrompt, device)
if err != nil {
t.Fatal(err)
}
if rgt, ok := target.(*referenceGraphTarget); ok {
if err := rgt.PushReference(context.Background(), desc, r, tag); err != nil {
t.Fatal(err)
}
if err := rgt.manager.Close(); err != nil {
t.Fatal(err)
}
} else {
t.Fatal("not testing based on a referenceGraphTarget")
}
// validate
if err = testutils.MatchPty(pty, device, donePrompt, desc.MediaType, "100.00%", desc.Digest.String()); err != nil {
t.Fatal(err)
}
}

func Test_referenceGraphTarget_Prompt(t *testing.T) {
// prepare
pty, device, err := testutils.NewPty()
if err != nil {
t.Fatal(err)
}
defer device.Close()
content := []byte("test")
desc := ocispec.Descriptor{
MediaType: "application/octet-stream",
Digest: digest.FromBytes(content),
Size: int64(len(content)),
}
// test
prompt := "prompt"
target, err := NewTarget(memory.New(), "action", "done", device)
if err != nil {
t.Fatal(err)
}
m := target.(*graphTarget).manager
if err := target.Prompt(desc, prompt); err != nil {
t.Fatal(err)
}
if err := m.Close(); err != nil {
t.Fatal(err)
}
// validate
if err = testutils.MatchPty(pty, device, prompt, desc.MediaType, "100.00%", desc.Digest.String()); err != nil {
t.Fatal(err)
}
}
2 changes: 1 addition & 1 deletion cmd/oras/root/attach.go
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ func runAttach(ctx context.Context, opts attachOptions) error {

// prepare push
var tracked track.GraphTarget
dst, tracked, err = getTrackedTarget(dst, opts.TTY)
dst, tracked, err = getTrackedTarget(dst, opts.TTY, "Uploading", "Uploaded ")
if err != nil {
return err
}
Expand Down
4 changes: 2 additions & 2 deletions cmd/oras/root/blob/fetch.go
Original file line number Diff line number Diff line change
Expand Up @@ -160,12 +160,12 @@ func (opts *fetchBlobOptions) doFetch(ctx context.Context, src oras.ReadOnlyTarg
}

if opts.TTY == nil {
// none tty output
// none TTY output
if _, err = io.Copy(writer, vr); err != nil {
return ocispec.Descriptor{}, err
}
} else {
// tty output
// TTY output
trackedReader, err := track.NewReader(vr, desc, "Downloading", "Downloaded ", opts.TTY)
if err != nil {
return ocispec.Descriptor{}, err
Expand Down
4 changes: 2 additions & 2 deletions cmd/oras/root/blob/push.go
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ func pushBlob(ctx context.Context, opts pushBlobOptions) (err error) {
}
func (opts *pushBlobOptions) doPush(ctx context.Context, t oras.Target, desc ocispec.Descriptor, r io.Reader) error {
if opts.TTY == nil {
// none tty output
// none TTY output
if err := display.PrintStatus(desc, "Uploading", opts.Verbose); err != nil {
return err
}
Expand All @@ -152,7 +152,7 @@ func (opts *pushBlobOptions) doPush(ctx context.Context, t oras.Target, desc oci
return display.PrintStatus(desc, "Uploaded ", opts.Verbose)
}

// tty output
// TTY output
trackedReader, err := track.NewReader(r, desc, "Uploading", "Uploaded ", opts.TTY)
if err != nil {
return err
Expand Down
4 changes: 2 additions & 2 deletions cmd/oras/root/cp.go
Original file line number Diff line number Diff line change
Expand Up @@ -180,12 +180,12 @@ func doCopy(ctx context.Context, src oras.ReadOnlyGraphTarget, dst oras.GraphTar
dst = tracked
extendedCopyOptions.OnCopySkipped = func(ctx context.Context, desc ocispec.Descriptor) error {
committed.Store(desc.Digest.String(), desc.Annotations[ocispec.AnnotationTitle])
return tracked.Prompt(desc, "Exists ", opts.Verbose)
return tracked.Prompt(desc, "Exists ")
}
extendedCopyOptions.PostCopy = func(ctx context.Context, desc ocispec.Descriptor) error {
committed.Store(desc.Digest.String(), desc.Annotations[ocispec.AnnotationTitle])
return display.PrintSuccessorStatus(ctx, desc, tracked, committed, func(desc ocispec.Descriptor) error {
return tracked.Prompt(desc, "Skipped", opts.Verbose)
return tracked.Prompt(desc, "Skipped")
})
}
}
Expand Down
Loading
Loading