Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

copyfile: remove callback #255

Closed
wants to merge 1 commit into from
Closed

copyfile: remove callback #255

wants to merge 1 commit into from

Conversation

skshetry
Copy link
Member

Let's remove the callback in all conditions.

The callbacks are part of fsspec, and they don't support callbacks in localfs either.
I plan on to remove tqdm and TqdmCallback from dvc-objects, which this was blocking.

Let's fix this properly if someone complains. :-)

@skshetry skshetry requested a review from efiop December 11, 2023 04:30
@codecov-commenter
Copy link

codecov-commenter commented Dec 11, 2023

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (b3a4b48) 63.71% compared to head (715ef2f) 64.16%.

Files Patch % Lines
src/dvc_objects/fs/local.py 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #255      +/-   ##
==========================================
+ Coverage   63.71%   64.16%   +0.44%     
==========================================
  Files          27       27              
  Lines        2067     2048      -19     
  Branches      325      319       -6     
==========================================
- Hits         1317     1314       -3     
+ Misses        690      675      -15     
+ Partials       60       59       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@efiop efiop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Completely removing feels like a bit too much. We should probably have a bytes-based (+number of objects) progress bar for all files, but that will still need some logic.

Let's fix this properly if someone complains. :-)

I don't think we can do it this way in this case, this is pretty important. We need some progress indication.

@efiop
Copy link
Contributor

efiop commented Dec 11, 2023

The callbacks are part of fsspec, and they don't support callbacks in localfs either.

That's not really a feature, it is just missing and needs to be added one way or another.

@skshetry
Copy link
Member Author

That's not really a feature, it is just missing and needs to be added one way or another.

Either you need to support callback or not. Unfortunately there is no middleground.

@efiop
Copy link
Contributor

efiop commented Dec 11, 2023

Either you need to support callback or not. Unfortunately there is no middleground.

I'm not sure I understand why it is an unresolvable problem though.

@skshetry
Copy link
Member Author

Looking through the responsibility of an API, if someone passes a callback and you decide to not create a callback, you are not honouring your part of promise here.

(Maybe there's a way to skip if we pass NoopCallback())

@efiop
Copy link
Contributor

efiop commented Dec 11, 2023

Looking through the responsibility of an API, if someone passes a callback and you decide to not create a callback, you are not honouring your part of promise here.

(Maybe there's a way to skip if we pass NoopCallback())

I agree and it probably just shouldn't create a callback on its own at all, but rather use the one provided by the user from a layer above.

@skshetry
Copy link
Member Author

I agree and it probably just shouldn't create a callback on its own at all, but rather use the one provided by the user from a layer above.

But you'd still be calling callback.{set_size,relative_update,absolute_update} though.

@efiop
Copy link
Contributor

efiop commented Dec 11, 2023

@skshetry Not if you pass a None/Noop as a callback (in case of Noop I mean somehow in a way that will avoid the overhead). The most expensive part IIRC was set_size, which we can give the honor to set back to user when he creates a callback, which he can decide not to if the file is small enough or for some other reason.

@skshetry
Copy link
Member Author

Can be closed, went with #256.

@skshetry skshetry closed this Dec 12, 2023
@skshetry skshetry deleted the copyfile-remove-callback branch December 12, 2023 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants