Skip to content

Commit

Permalink
guide: add Configuration guide (#4379)
Browse files Browse the repository at this point in the history
* guide: add Config guide

per #340

* Include basic remote storage config info and...

and/or link to
https://dvc.org/doc/user-guide/data-management/remote-storage

* Update content/docs/command-reference/config.md

Co-authored-by: Dave Berenbaum <[email protected]>

* Restyled by prettier (#4383)

Co-authored-by: Restyled.io <[email protected]>

* guide: update links to Config info. (#4380)

* guide: Update links to config sections

Previously in the cmd ref, now in a new guide

* guide: some more links related to `dvc config` instances

* guide: update links from `dvc config` (ref) to new guide

* guide: clarify about other config file locations

per #4379 (review)

* guide: refinements

per #4379 (review)
and #4379 (comment)

* Update content/docs/user-guide/project-structure/configuration.md

---------

Co-authored-by: Dave Berenbaum <[email protected]>

---------

Co-authored-by: Dave Berenbaum <[email protected]>
Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com>
Co-authored-by: Restyled.io <[email protected]>
  • Loading branch information
4 people authored Mar 16, 2023
1 parent 8b90e10 commit 29f5ebd
Show file tree
Hide file tree
Showing 23 changed files with 551 additions and 404 deletions.
2 changes: 1 addition & 1 deletion content/docs/command-reference/add.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ other DVC commands), a few actions are taken under the hood:
[remote storage]: /doc/user-guide/data-management/remote-storage
[structure of cache directory]:
/doc/user-guide/project-structure/internal-files#structure-of-the-cache-directory
[`core.autostage`]: /doc/command-reference/config#core
[`core.autostage`]: /doc/user-guide/project-structure/configuration#core

Summarizing, the result is that the target data is replaced by small `.dvc`
files that can be easily tracked with Git.
Expand Down
24 changes: 14 additions & 10 deletions content/docs/command-reference/checkout.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,15 +45,18 @@ after `git checkout`. See the
for more details.

By default, this command tries not make copies of cached files in the workspace,
using reflinks instead when supported by the file system (refer to
[File link types](/doc/user-guide/data-management/large-dataset-optimization#file-link-types-for-the-dvc-cache)).
The next linking strategy default value is `copy` though, so unless other file
link types are manually configured in `cache.type` (using `dvc config`), files
will be copied. Keep in mind that having file copies doesn't present much of a
negative impact unless the project uses very large data (several GBs or more).
But leveraging file links is crucial with large files, for example when checking
out a 50Gb file by copying might take a few minutes whereas, with links,
restoring any file size will be almost instantaneous.
using reflinks instead when supported by the file system (refer to [File link
types]). The next linking strategy default value is `copy` though, so unless
other file link types are manually configured in [`cache.type`]), files will be
copied. Keep in mind that having file copies doesn't present much of a negative
impact unless the project uses very large data (several GBs or more). But
leveraging file links is crucial with large files, for example when checking out
a 50Gb file by copying might take a few minutes whereas, with links, restoring
any file size will be almost instantaneous.

[File link types]:
/doc/user-guide/data-management/large-dataset-optimization#file-link-types-for-the-dvc-cache
[`cache.type`]: /doc/user-guide/project-structure/configuration#cache

> When linking files takes longer than expected (10 seconds for any one file)
> and `cache.type` is not set, a warning will be displayed reminding users about
Expand Down Expand Up @@ -95,7 +98,8 @@ situation. In some cases, the data can be pulled from [remote storage] using

- `--relink` - ensures the file linking strategy (`reflink`, `hardlink`,
`symlink`, or `copy`) for all data in the workspace is consistent with the
project's [`cache.type`](/doc/command-reference/config#cache). This is
project's
[`cache.type`](/doc/user-guide/project-structure/configuration#cache). This is
achieved by restoring **all data files or directories** referenced in current
DVC files (regardless of whether the files/dirs were already present).

Expand Down
Loading

0 comments on commit 29f5ebd

Please sign in to comment.