Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support copying to/from non-regular files (character devices, named pipes, etc). #813

Merged
merged 8 commits into from
Jun 3, 2019

Conversation

serhiy-storchaka
Copy link
Contributor

In particularly this allows to output the file to the stdout or get the input from the stdin:

neuro storage cp storage://~/file.txt /dev/stdout
neuro storage cp /dev/stdin storage://~/file.txt

See #281.

@serhiy-storchaka serhiy-storchaka changed the title Support copying to/from non-regular files (character devices, named files, etc). Support copying to/from non-regular files (character devices, named pipes, etc). May 28, 2019
@serhiy-storchaka serhiy-storchaka force-pushed the copy-non-regular-file branch from 8823852 to 6fc1552 Compare May 28, 2019 08:12
@dalazx
Copy link
Contributor

dalazx commented May 28, 2019

the tests are failing on Windows.

@codecov
Copy link

codecov bot commented May 28, 2019

Codecov Report

Merging #813 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #813      +/-   ##
==========================================
- Coverage   90.25%   90.25%   -0.01%     
==========================================
  Files          38       38              
  Lines        3347     3344       -3     
  Branches      484      482       -2     
==========================================
- Hits         3021     3018       -3     
  Misses        238      238              
  Partials       88       88
Impacted Files Coverage Δ
neuromation/api/storage.py 98.98% <100%> (-0.02%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d5a04bd...26d0a17. Read the comment docs.

@codecov
Copy link

codecov bot commented May 28, 2019

Codecov Report

Merging #813 into master will decrease coverage by 1.03%.
The diff coverage is 76.47%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #813      +/-   ##
==========================================
- Coverage   89.71%   88.67%   -1.04%     
==========================================
  Files          38       38              
  Lines        3460     3462       +2     
  Branches      493      493              
==========================================
- Hits         3104     3070      -34     
- Misses        262      291      +29     
- Partials       94      101       +7
Impacted Files Coverage Δ
neuromation/api/storage.py 97.08% <76.47%> (-1.92%) ⬇️
neuromation/cli/image.py 51.92% <0%> (-48.08%) ⬇️
neuromation/api/images.py 86.91% <0%> (-5.61%) ⬇️
neuromation/cli/config.py 65% <0%> (-1.35%) ⬇️
neuromation/api/config_factory.py 86.66% <0%> (-1.34%) ⬇️
neuromation/cli/utils.py 93.3% <0%> (-0.9%) ⬇️
neuromation/api/jobs.py 91.43% <0%> (-0.35%) ⬇️
neuromation/api/url_utils.py 87.71% <0%> (ø) ⬆️
neuromation/cli/formatters/jobs.py 93.47% <0%> (ø) ⬆️
neuromation/cli/main.py 62.82% <0%> (+2.09%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef95761...4701306. Read the comment docs.

@@ -78,7 +78,7 @@ def normalize_local_path_uri(uri: URL) -> URL:
path = path.expanduser()
if path.parents and str(path.parents[0]).startswith("~"):
raise ValueError(f"Cannot expand user for {uri}")
path = path.absolute()
path = Path.cwd() / path
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is a bug introduced in previous changes. I did not notice the failure on Windows among the number of random errors.

CHANGELOG.D/813.feature Outdated Show resolved Hide resolved
@@ -0,0 +1 @@
`neuro storage cp` now supports copying to/from non-regular files like character devices and named pipes. In particular this allows to output the file to the stdout or get the input from the stdin (`/dev/stdout` and `/dev/stdin` on Linux, `CON` on Windows).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we please also mention this in the command documentation?

@asvetlov asvetlov merged commit d39c651 into master Jun 3, 2019
@asvetlov asvetlov deleted the copy-non-regular-file branch June 3, 2019 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants