cli: extend logs of index file construction for upload-bin
#3662
Conversation
These logs are needed to track the progress of uploaders. Signed-off-by: Anna Shaleva <[email protected]>
Signed-off-by: Anna Shaleva <[email protected]>
AnnaShaleva
requested review from
AliceInHunterland
and removed request for
roman-khimov
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3662 +/- ##
==========================================
+ Coverage 83.13% 83.14% +0.01%
==========================================
Files 334 334
Lines 46563 46563
==========================================
+ Hits 38708 38715 +7
+ Misses 6278 6273 -5
+ Partials 1577 1575 -2 ☔ View full report in Codecov by Sentry. |
|
By the way, maybe |
AnnaShaleva
force-pushed
the
improve-uploadbin-log
branch
from
e0f0bf0 to
3ac1129
Compare
I also thought about it, but it uploads not only blocks, but also index files. So naming is hard.
It's an analogue of existing |
Follow the style of other files in the `util` package and existing `dump_bin.go`. Signed-off-by: Anna Shaleva <[email protected]>
AnnaShaleva
force-pushed
the
improve-uploadbin-log
branch
from
3ac1129 to
a85f3ce
Compare
| @@ -425,14 +425,15 @@ func uploadIndexFiles(ctx *cli.Context, p *pool.Pool, containerID cid.ID, accoun | |||
| for idx := range indexFileSize { | |||
| if _, ok := processedIndices.Load(idx); !ok { | |||
| count++ | |||
| fmt.Fprintf(ctx.App.Writer, "Index file %d: fetching missing block %d\n", i, i*indexFileSize+idx) | |||
There was a problem hiding this comment.
Won't it be too chatty?
There was a problem hiding this comment.
It will, but currently there's completely no way to track the progress of "gaps" completion. Everything what I have is just one line:
2024-11-05 13:39:11.228 Index file X generated, checking for the missing blocks...
which is hanging in the air for several hours. This log allows us to track the progress and (in bad cases) to learn which objects are missing from the initial SEARCH request, it may help during NeoFS-related issues debugging.
There was a problem hiding this comment.
Hence I suggest to keep these logs for now.
These logs are needed to track the progress of uploaders. Based on these logs we'll try to optimize missing blocks fetching during index files construction.