Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

guide: absorb What is DVC? into other existing docs, et al. #1581

Merged
merged 44 commits into from
Aug 10, 2020
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
6690907
guide: What is DVC? -> into UG index
jorgeorpinel Jul 15, 2020
ce72b11
how-to: create section with questions from WID / Collab Issues
jorgeorpinel Jul 15, 2020
b466986
Merge branch 'master' into guide/what-is-dvc
jorgeorpinel Jul 20, 2020
3f0b0f0
how-to: make subsection of the user-guide, and
jorgeorpinel Jul 20, 2020
5a5901b
guide: hide Best Practices how to for now
jorgeorpinel Jul 20, 2020
a94d9f8
guide: rename how to and best practices title
jorgeorpinel Jul 20, 2020
92ae254
guide: What->Why in index to avoid redundancy with What section
jorgeorpinel Jul 20, 2020
d7762e6
guide: concepts->principles in What is DVC?
jorgeorpinel Jul 20, 2020
82554fe
guide: move troubleshooting inside How To
jorgeorpinel Jul 20, 2020
05a7a7d
guide: collapse What is DVC? into single doc, and
jorgeorpinel Jul 20, 2020
3e46fca
guide: fix redirect test for troubleshooting how to
jorgeorpinel Jul 20, 2020
93cc607
guide: revise What is DVC? up to Core Principles and
jorgeorpinel Jul 20, 2020
8a7c086
guide: finish revising What is DVC?
jorgeorpinel Jul 20, 2020
c30e966
guide: more updates to What is DVC? (per 1.x) and
jorgeorpinel Jul 20, 2020
733593c
guide: review intro and reorg Related Technologies
jorgeorpinel Jul 20, 2020
28226d6
Merge branch 'master' into guide/what-is-dvc
jorgeorpinel Jul 22, 2020
7f421dd
guide: add Questions header to best practices (hidden)
jorgeorpinel Jul 22, 2020
d390a6f
guide: hide GAPI PP
jorgeorpinel Jul 22, 2020
c97e93b
guide: revise Git-LFS section of related techs
jorgeorpinel Jul 23, 2020
30d38df
guide: revise all Git* related techs
jorgeorpinel Jul 23, 2020
a450d39
guide: revise remaining related techs
jorgeorpinel Jul 23, 2020
a93f24f
guide: remove img from basic concepts
jorgeorpinel Jul 23, 2020
1a6948e
guide: move troubleshooting back out of How To
jorgeorpinel Jul 23, 2020
3e7b18a
Merge branch 'master' into guide/what-is-dvc
jorgeorpinel Aug 3, 2020
a556e6c
cases: move Why DVC? to Use Cases index and
jorgeorpinel Aug 4, 2020
eb5fbf9
guide: move Basic Principles from What is DVC? into Basic Concepts guide
jorgeorpinel Aug 4, 2020
c476f60
guide: remove "User Manual" term from index
jorgeorpinel Aug 4, 2020
3508a19
nav: remove ... from How To entry
jorgeorpinel Aug 4, 2020
2842e3a
tests: finis rolling back troubleshooting guide move
jorgeorpinel Aug 4, 2020
02c2b01
Merge branch 'master' into guide/what-is-dvc
jorgeorpinel Aug 8, 2020
7411f53
cases: fix a link to related techs guide
jorgeorpinel Aug 8, 2020
9c75ae8
Merge branch 'master' into guide/what-is-dvc
jorgeorpinel Aug 8, 2020
98ffea3
guide: propper structure in related techs
jorgeorpinel Aug 9, 2020
95521e6
guide: update remote storage core concept in what is dvc
jorgeorpinel Aug 9, 2020
fbd7e96
guide: improve Core Features of What is DVC?
jorgeorpinel Aug 9, 2020
86fbf43
guide: simplify data versioning core feature
jorgeorpinel Aug 9, 2020
de43edd
guide: update What is DVC? intro
jorgeorpinel Aug 9, 2020
bebd665
guide: simplify Core Features in What is DVC?
jorgeorpinel Aug 9, 2020
d8a71f8
guide: features before concepts (index)
jorgeorpinel Aug 9, 2020
722cbb3
guide: review term "features" in basic concepts
jorgeorpinel Aug 9, 2020
e6d5f78
guide: undo starting How To subsection
jorgeorpinel Aug 10, 2020
ca9203a
guide: undo changes to troubleshooting
jorgeorpinel Aug 10, 2020
eebb2e6
guide: a few more copy edits for What is DVC
jorgeorpinel Aug 10, 2020
c720552
guide: remove Basic Concepts page
jorgeorpinel Aug 10, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions content/docs/how-to/best-practices.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Best Practices
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved

Data scientists, engineers, or managers may already know or can easily find
answers to some of these questions. However, the variety of answers and
approaches makes data science collaboration a nightmare. **A systematic approach
is required.**

## Source code and data versioning

- How do you avoid discrepancies between
[revisions](https://git-scm.com/docs/revisions) of source code and versions of
data files, when the data cannot fit into a traditional repository?

## Experiment time log

- How do you track which of your
[hyperparameter](<https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)>)
changes contributed the most to producing or improving your target
[metric](/doc/command-reference/metrics)? How do you monitor the degree of
each change?

## Navigating through experiments

- How do you recover a model from last week without wasting time waiting for the
model to retrain?

- How do you quickly switch between a large dataset and a small subset without
modifying source code?

## Reproducibility

- How do you run a model's evaluation process again without retraining the model
and preprocessing a raw dataset?

## Managing and sharing large data files

- How do you share models trained in a GPU environment with colleagues who don't
have access to a GPU?

- How do you share the entire 147 GB of your ML project, with all of its data
sources, intermediate data files, and models?
16 changes: 11 additions & 5 deletions content/docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -86,11 +86,7 @@
"slug": "what-is-dvc",
"label": "What is DVC?",
"source": "what-is-dvc/index.md",
"children": [
"collaboration-issues",
"core-features",
"related-technologies"
]
"children": ["core-features", "related-technologies"]
},
{
"label": "DVC Files and Directories",
Expand Down Expand Up @@ -144,6 +140,16 @@
}
]
},
{
"slug": "how-to",
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
"source": false,
"children": [
{
"slug": "best-practices",
"source": "how-to/best-practices.md"
}
]
},
{
"slug": "command-reference",
"source": "command-reference/index.md",
Expand Down
19 changes: 15 additions & 4 deletions content/docs/user-guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,22 @@

Our guides describe the main DVC concepts and features comprehensively,
explaining when and how to use them, as well as connections between them. These
guides don't focus on specific scenarios, but have a general scope – like a user
manual. Their topics range from more technical foundations, impacting more parts
of DVC, to more advanced and specific things you can do. We also include a few
guides related to contributing to
guides don't focus on specific scenarios, but have a general scope – like a
_user manual_. Their topics range from more technical foundations, impacting
more parts of DVC, to more advanced and specific things you can do. We also
include a few guides related to contributing to
[this open-source project](https://github.com/iterative/dvc).

## What is DVC?

Even with all the success we've seen today in machine learning (ML),
specifically deep learning and its applications in business, the data science
community still lacks good practices for organizing their projects and
collaborating effectively. This is a critical challenge: we need to evolve
towards ML algorithms and methods no longer being tribal knowledge and making
them easy to implement, reuse, and manage.

---

Please choose from the navigation sidebar to the left, or click the `Next`
button below ↘
53 changes: 0 additions & 53 deletions content/docs/user-guide/what-is-dvc/collaboration-issues.md

This file was deleted.