Skip to content

Commit

Permalink
guide: move "local remotes" to Remotes (index page) and
Browse files Browse the repository at this point in the history
update admonitions and links
  • Loading branch information
jorgeorpinel committed Feb 2, 2023
1 parent b2cc99c commit d07f2bf
Show file tree
Hide file tree
Showing 6 changed files with 70 additions and 76 deletions.
20 changes: 13 additions & 7 deletions content/docs/command-reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -250,9 +250,8 @@ location. A [DVC remote](/doc/command-reference/remote) name is used (instead of
the URL) because often it's necessary to configure authentication or other
connection settings, and configuring a remote is the way that can be done.

- `cache.local` - name of a _local remote_ to use as external cache (refer to
`dvc remote` for more info. on "local remotes".) This will overwrite the value
in `cache.dir` (see `dvc cache dir`).
- `cache.local` - name of a [local remote] to use as external cache. This will
overwrite the value in `cache.dir` (see `dvc cache dir`).

- `cache.s3` - name of an Amazon S3 remote to use as external cache.

Expand All @@ -265,10 +264,17 @@ connection settings, and configuring a remote is the way that can be done.
- `cache.webhdfs` - name of an HDFS remote with WebHDFS enabled to use as
external cache.

> ⚠️ Avoid using the same [remote storage](/doc/command-reference/remote) used
> for `dvc push` and `dvc pull` as external cache, because it may cause file
> hash overlaps: the hash of an external <abbr>output</abbr> could collide with
> that of a local file with different content.
<admon type="warn">

Avoid using the same [remote storage](/doc/command-reference/remote) used for
`dvc push` and `dvc pull` as external cache, because it may cause file hash
overlaps: the hash of an external <abbr>output</abbr> could collide with that
of a local file with different content.

</admon>

[local remote]:
/doc/user-guide/data-management/remote-storage#file-systems-local-remotes

### exp

Expand Down
42 changes: 1 addition & 41 deletions content/docs/command-reference/remote/add.md
Original file line number Diff line number Diff line change
Expand Up @@ -325,47 +325,7 @@ $ dvc remote add -d myremote \
</details>

<details>

### local remote

A "local remote" is a directory in the machine's file system. Not to be confused
with the `--local` option of `dvc remote` (and other config) commands!

> While the term may seem contradictory, it doesn't have to be. The "local" part
> refers to the type of location where the storage is: another directory in the
> same file system. "Remote" is how we call storage for <abbr>DVC
> projects</abbr>. It's essentially a local backup for data tracked by DVC.
Using an absolute path (recommended):

```cli
$ dvc remote add -d myremote /tmp/dvcstore
$ cat .dvc/config
...
['remote "myremote"']
url = /tmp/dvcstore
...
```

> Note that the absolute path `/tmp/dvcstore` is saved as is.
Using a relative path. It will be resolved against the current working
directory, but saved **relative to the config file location**:

```cli
$ dvc remote add -d myremote ../dvcstore
$ cat .dvc/config
...
['remote "myremote"']
url = ../../dvcstore
...
```

> Note that `../dvcstore` has been resolved relative to the `.dvc/` dir,
> resulting in `../../dvcstore`.
</details>
<!-- ### local remote -->

## Example: Customize an S3 remote

Expand Down
12 changes: 4 additions & 8 deletions content/docs/command-reference/remote/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,16 +82,12 @@ edited manually.

## Example: Add a default local remote

<details>
<admon type="tip">

### What is a "local remote" ?
Learn more about
[local remotes](/doc/user-guide/data-management/remote-storage#file-systems-local-remotes).

While the term may seem contradictory, it doesn't have to be. The "local" part
refers to the type of location where the storage is: another directory in the
same file system. "Remote" is how we call storage for <abbr>DVC projects</abbr>.
It's essentially a local backup for data tracked by DVC.

</details>
</admon>

We use the `-d` (`--default`) option of `dvc remote add` for this:

Expand Down
16 changes: 4 additions & 12 deletions content/docs/command-reference/remote/list.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,18 +39,7 @@ local config files (in that order).

## Examples

For simplicity, let's add a default local remote:

<details>

### What is a "local remote" ?

While the term may seem contradictory, it doesn't have to be. The "local" part
refers to the type of location where the storage is: another directory in the
same file system. "Remote" is how we call storage for <abbr>DVC projects</abbr>.
It's essentially a local backup for data tracked by DVC.

</details>
For simplicity, let's add a default [local remote]:

```cli
$ dvc remote add -d myremote /path/to/remote
Expand All @@ -65,3 +54,6 @@ myremote /path/to/remote
```

The list will also include any previously added remotes.

[local remote]:
/doc/user-guide/data-management/remote-storage#file-systems-local-remotes
8 changes: 3 additions & 5 deletions content/docs/start/data-management/data-versioning.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,12 +141,10 @@ $ git commit .dvc\config -m "Configure local remote"
</tab>
</toggle>

<admon type="info">
<admon type="tip">

While the term "local remote" may seem contradictory, it doesn't have to be. The
"local" part refers to the type of location: another directory in the file
system. "Remote" is what we call storage for <abbr>DVC projects</abbr>. It's
essentially a local data backup.
Learn more about
[local remotes](/doc/user-guide/data-management/remote-storage#file-systems-local-remotes).

</admon>

Expand Down
48 changes: 45 additions & 3 deletions content/docs/user-guide/data-management/remote-storage/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,48 @@ team.
- HDFS & WebHDFS
- HTTP
- WebDAV
- Local directories, mounted drives; Like `rsync`
> Includes network resources e.g. network-attached storage (NAS) or other
> external devices

## File systems (local remotes)

<admon type="tip">

Not related to the `--local` option of `dvc remote` and `dvc config`!

</admon>

You can also use system directories, mounted drives, network resources e.g.
network-attached storage (NAS), and other external devices as storage. We call
all these "local remotes".

<admon type="info">

Here, the word "local" refers to where the storage is found: typically another
directory in the same file system. And "remote" is how we call storage for
<abbr>DVC projects</abbr>.

</admon>

Using an absolute path (recommended because it's saved as-is in DVC config):

```cli
$ dvc remote add -d myremote /tmp/dvcstore
```

```ini
# .dvc/config
['remote "myremote"']
url = /tmp/dvcstore
```

When using a relative path, it will be saved **relative to the config file
location**, but resolved against the current working directory.

```cli
$ dvc remote add -d myremote ../dvcstore
```

```ini
# .dvc/config
['remote "myremote"']
url = ../../dvcstore
```

0 comments on commit d07f2bf

Please sign in to comment.