Add the --update option to "storage cp".

[serhiy-storchaka]

neuro storage cp --update should copy only when the source file is newer than the destination file or when the destination file is missing.

[serhiy-storchaka]

Issues:

• We cannot just compare the modification time, because local and remote modification time can be out of sync. Needs to estimate the margin and compare with a threshold.

• Currently uploading with --update is slower than without it, because getting the remote directory listing is expensive. We should get a listing only if the remote directory is newer than the local one (its stat is already known).

[codecov]

Codecov Report

Merging #1007 into master will decrease coverage by 1.24%.
The diff coverage is 34.04%.

@@            Coverage Diff             @@
##           master    #1007      +/-   ##
==========================================
- Coverage   88.37%   87.12%   -1.25%     
==========================================
  Files          37       37              
  Lines        4566     4551      -15     
  Branches      681      680       -1     
==========================================
- Hits         4035     3965      -70     
- Misses        414      457      +43     
- Partials      117      129      +12

Impacted Files	Coverage Δ
neuromation/cli/storage.py	53.12% <100%> (-0.3%)	⬇️
neuromation/api/storage.py	89.07% <26.19%> (-7.92%)	⬇️
neuromation/api/images.py	77.51% <0%> (-13.18%)	⬇️
neuromation/cli/formatters/config.py	88.88% <0%> (-11.12%)	⬇️
neuromation/api/url_utils.py	87.87% <0%> (-4.55%)	⬇️
neuromation/api/config_factory.py	89.1% <0%> (-1.29%)	⬇️
neuromation/api/jobs.py	92.5% <0%> (-1.14%)	⬇️
neuromation/cli/formatters/jobs.py	95.47% <0%> (-0.86%)	⬇️
neuromation/cli/utils.py	83.44% <0%> (-0.67%)	⬇️
neuromation/cli/main.py	64.35% <0%> (-0.5%)	⬇️
... and 3 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4a67df7...0c473ef. Read the comment docs.

[asvetlov]

lgtm, but please use monotonic time clock just for the case

[asvetlov]

I sec I guess?

[serhiy-storchaka]

Yes, 1 sec. Tests set it even to 0 sec.