Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

how-to: setup a shared cache (extracted from use cases) #2482

Merged
merged 27 commits into from
Jun 29, 2021
Merged

how-to: setup a shared cache (extracted from use cases) #2482

merged 27 commits into from
Jun 29, 2021

Conversation

jorgeorpinel
Copy link
Contributor

@jorgeorpinel jorgeorpinel commented May 16, 2021 •
edited
Loading

REPURPOSED to close #2487

Would close (no longer) #654 (comment)
And (not yet) #820

@jorgeorpinel
cases&how: generalizing shared dev server case and
9bca7fe 
and extract guided info to new how-to
@shcheklein shcheklein temporarily deployed to dvc-org-cases-shareing--sizh7a May 16, 2021 23:52 Inactive
@jorgeorpinel

This comment has been minimized.

Sign in to view
@jorgeorpinel jorgeorpinel changed the title Generalizing shared dev server case and Generalizing Shared Dev Server case and May 16, 2021
@jorgeorpinel jorgeorpinel self-assigned this May 16, 2021
@jorgeorpinel jorgeorpinel mentioned this pull request May 17, 2021
Closed
8 tasks
@jorgeorpinel
cases: rewrite intro to Sharing Res and
c776a77 
draft a first section on storage
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-cases-shareing--sizh7a May 18, 2021 05:44 Inactive
@jorgeorpinel jorgeorpinel mentioned this pull request May 18, 2021
Closed
shcheklein
shcheklein previously requested changes May 18, 2021
View reviewed changes
Copy link
Member

@shcheklein shcheklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good iteration of the use case - close to what we've been doing with other use cases. A bit of polishing is still required, but it's a good one indeed! We are close.

@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-cases-shareing--sizh7a May 19, 2021 01:51 Inactive
@jorgeorpinel jorgeorpinel changed the title cases: Generalizing Shared Dev Server, and guide: extract shared cache info from use cases Jun 28, 2021
@jorgeorpinel jorgeorpinel changed the title guide: extract shared cache info from use cases how-to: extract shared cache info from use cases to guide Jun 28, 2021
@jorgeorpinel jorgeorpinel removed a link to an issue Jun 28, 2021
use-cases: revise so they're more high level "landing pages" #820
Closed
8 tasks
@jorgeorpinel
Copy link
Contributor Author

@shcheklein I've repurposed this to only move the shared cache info to the guide for now. The existing use case will remain untouched but no longer linked from everywhere. Please review content/docs/user-guide/how-to/share-a-dvc-cache.md when you have a chance. All other changes are links and sidebar.

@jorgeorpinel jorgeorpinel requested review from shcheklein and removed request for shcheklein June 28, 2021 23:32
@jorgeorpinel jorgeorpinel changed the title how-to: extract shared cache info from use cases to guide how-to: setup a shared cache (extracted from use cases) Jun 28, 2021
@jorgeorpinel jorgeorpinel mentioned this pull request Jun 28, 2021
Merged
5 tasks
shcheklein
shcheklein reviewed Jun 29, 2021
View reviewed changes
content/docs/command-reference/gc.md Outdated
specify a list of them (each project is a path) to keep data that is currently
referenced from them.
shared among different projects (e.g. as described
[here](/doc/use-cases/sharing-resources-efficiently)), this option can be used
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not part of this PR, right? should the use case one go first?

This comment was marked as resolved.

Sign in to view

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

should the use case one go first?

Hmm the order isn't that important I think. Since this one is ready, why not go for it. This way we don't need to worry about where to put the shared cache info. later when the shared dev server use case changes completely.

shcheklein
shcheklein reviewed Jun 29, 2021
View reviewed changes
content/docs/user-guide/how-to/share-a-dvc-cache.md Outdated

# How to Share a DVC Cache

There are 2 main reasons to setup a shared <abbr>DVC cache</abbr>:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

both items are pretty hard to read to be honest

so, we have a few cases people would use this:

  • one large machine (e.g. with multiple GPUs) and one storage on it and people do the same or multiple projects and we want to avoid duplication (no copies) and save time (no copy)
  • one large NAS () and people attach it to their machines - again no copies - fast, doesn't take extra space, fits even if there are no space (ability to work with really large data)

what else am I missing? can we make description more explicit/simpler?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Updated, PTAL.

jorgeorpinel added a commit that referenced this pull request Jun 29, 2021
@jorgeorpinel
cases: remove references to non-existent doc
0611d2a 
per #2482 (review)
and #2482 (review)
@jorgeorpinel jorgeorpinel requested a review from shcheklein June 29, 2021 04:28
shcheklein
shcheklein requested changes Jun 29, 2021
View reviewed changes
Copy link
Member

@shcheklein shcheklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are two images - do we use those?

Do we need to setup a redirect?

@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-cases-shareing--2pfth4 June 29, 2021 05:38 Inactive
@jorgeorpinel
cases: remove unnecessary images
94c6cbd 
per static/img/resource-pool.png
@jorgeorpinel jorgeorpinel temporarily deployed to dvc-org-cases-shareing--2pfth4 June 29, 2021 05:47 Inactive
@jorgeorpinel
Copy link
Contributor Author

jorgeorpinel commented Jun 29, 2021 •
edited
Loading

🤦 removed the images.

No doc is removed yet so I see no need for a redirect here. I did change all the relevant links to the how-to for now.

#2587 will have the redirect though.

@jorgeorpinel
Copy link
Contributor Author

Sanity checked https://dvc-org-cases-shareing--2pfth4.herokuapp.com/doc/user-guide/how-to/share-a-dvc-cache 🚀

@jorgeorpinel jorgeorpinel merged commit b83b508 into master Jun 29, 2021
@jorgeorpinel jorgeorpinel deleted the cases/shareing-res branch July 15, 2021 00:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shcheklein shcheklein shcheklein approved these changes

Assignees

@jorgeorpinel jorgeorpinel

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

guide: extract shared cache info from shared dev server case
2 participants
@jorgeorpinel @shcheklein