Merge pull request #8 from iterative/dvc-updates-05-05

Incorporate dvc master and update docs (05/05)

packages/example/content/docs/command-reference/diff.md

@@ -119,7 +119,7 @@ $ dvc diff
 
 ## Example: Comparing workspace with arbitrary commits
 
-<details>
+<details id="example-arbitrary-commits-set-up">
 
 ### Click and expand to set up the example
 
@@ -149,7 +149,7 @@ files summary: 1 added, 0 deleted, 0 modified
 
 ## Example: Comparing tags or branches
 
-<details>
+<details id="example-tags-branches-set-up">
 
 ### Click and expand to set up the example
 
@@ -223,7 +223,7 @@ It outputs:
 
 ## Example: Renamed files
 
-<details>
+<details id="example-renamed-files-set-up">
 
 ### Click and expand to set up the example
 

packages/example/content/docs/command-reference/exp/pull.md

@@ -6,8 +6,8 @@ data from a `dvc remote`.
 ## Synopsis
 
 ```usage
-usage: dvc exp pull [-h] [-q | -v] [-f] [--no-cache]
-                    [-r <name>] [-j <number>] [--run-cache]
+usage: dvc exp pull [-h] [-q | -v] [-A] [--rev <commit>] [-n <num>] [-f]
+                    [--no-cache] [-r <name>] [-j <number>] [--run-cache]
                     git_remote experiment
 
 positional arguments:
@@ -43,6 +43,16 @@ all <abbr>cached</abbr> data associated with the experiment to DVC
 
 ## Options
 
+- `-A`, `--all-commits` - pull all experiments in the repository (overrides
+  `--rev` and `--num`).
+
+- `--rev <commit>` - pull experiments derived from the specified `<commit>` as
+  baseline.
+
+- `-n <num>`, `--num <num>` - show experiments from the last `num` commits
+  (first parents) starting from the `--rev` baseline. Give a negative value to
+  include all first-parent commits (similar to `git log -n`).
+
 - `-f`, `--force` - rewrite the `experiment` commit if it already exists in the
   local repo. Equivalent to `git push --force` (rewrites history)
 

packages/example/content/docs/command-reference/exp/push.md

@@ -6,8 +6,8 @@ to a `dvc remote`.
 ## Synopsis
 
 ```usage
-usage: dvc exp push [-h] [-q | -v] [-f] [--no-cache]
-                    [-r <name>] [-j <number>] [--run-cache]
+usage: dvc exp push [-h] [-q | -v] [-A] [--rev <commit>] [-n <num>] [-f]
+                    [--no-cache] [-r <name>] [-j <number>] [--run-cache]
                     git_remote experiment
 
 positional arguments:
@@ -40,6 +40,16 @@ This command will also try to [push](/doc/command-reference/push) all
 
 ## Options
 
+- `-A`, `--all-commits` - push all experiments in the repository (overrides
+  `--rev` and `--num`).
+
+- `--rev <commit>` - push experiments derived from the specified `<commit>` as
+  baseline.
+
+- `-n <num>`, `--num <num>` - show experiments from the last `num` commits
+  (first parents) starting from the `--rev` baseline. Give a negative value to
+  include all first-parent commits (similar to `git log -n`).
+
 - `-f`, `--force` - rewrite the `experiment` commit if it already exists in the
   Git remote. Equivalent to `git push --force` (rewrites history)
 

packages/example/content/docs/command-reference/exp/remove.md

@@ -5,7 +5,8 @@ Delete specific experiments from the <abbr>project</abbr>.
 ## Synopsis
 
 ```usage
-usage: dvc exp remove [-h] [-q | -v] [--queue | -A | -g <git_remote>]
+usage: dvc exp remove [-h] [-q | -v] [-A] [--rev <commit>] [-n <num>]
+                      [--queue | -g <git_remote>]
                       [<name> [<name> ...]]
 
 positional arguments:
@@ -29,6 +30,13 @@ With `--queue`, the list of experiments awaiting execution is cleared instead.
 - `-A`, `--all` - remove all experiments that have been run. Use `--queue` to
   remove queued ones.
 
+- `--rev <commit>` - remove experiments derived from the specified `<commit>` as
+  baseline.
+
+- `-n <num>`, `--num <num>` - show experiments from the last `num` commits
+  (first parents) starting from the `--rev` baseline. Give a negative value to
+  include all first-parent commits (similar to `git log -n`).
+
 - `-g`, `--git-remote` - Name or URL of the Git remote to remove the experiment
   from
 

packages/example/content/docs/command-reference/exp/run.md

@@ -7,7 +7,7 @@ Run or resume a
 
 ```usage
 usage: dvc exp run [-h] [-q | -v] [-f]
-                   { repro options ... }
+                   { repro options ... } [-n <name>]
                    [-S [<filename>:]<params_list>]
                    [--queue] [--run-all] [-j <number>] [--temp]
                    [-r <experiment_rev>] [--reset]

packages/example/content/docs/command-reference/gc.md

@@ -44,8 +44,29 @@ If the `--cloud` (`-c`) flag is used, this command deletes unused data from the
 to deleting it from the local DVC cache. To specify a DVC remote to delete from,
 use the `--remote` (`-r`) option.
 
-> ⚠️ Danger: cloud deletion is irreversible unless there is another DVC remote
-> or a manual backup with the same data.
+<admon type="warn">
+
+Cloud deletion is irreversible unless there is another DVC remote or a manual
+backup with the same data.
+
+</admon>
+
+### Cleaning shared cache (or remote)
+
+If a [cache is shared] among different projects that track some of the same
+files, using `dvc gc` in one project will break those overlapping data links in
+the other projects.
+
+To prevent this, use the `--projects` (`-p`) option. It takes one or more paths
+to the DVC project(s) whose data should be preserved. Make sure that all the
+commits and branches that reference files you want to keep have been pulled in
+those other projects first.
+
+For example, if we have several projects with some overlapping files and we'd
+like to collect all the data that's only used in one of them (e.g. if we no
+longer need that projects), we would first clone all the other projects, fetch
+all their branches, and pass their paths to the `dvc gc -p` command from the
+project we want to clear.
 
 ## Options
 
@@ -75,17 +96,15 @@ use the `--remote` (`-r`) option.
 
   > \* Not including [DVC experiments]
 
-[dvc experiments]: /doc/user-guide/experiment-management#experiments
-
 - `--all-experiments` keep cached objects referenced in all [DVC experiments],
   as well as in the workspace (implying `-w`). This preserves the project's
   [experimental](/doc/user-guide/experiment-management) data (including
   checkpoints). See also `dvc exp gc`.
 
-- `-p <paths>`, `--projects <paths>` - if a single remote or a single
-  [cache is shared](/doc/user-guide/how-to/share-a-dvc-cache) among different
-  projects, this option can be used to specify a list of them (each project is a
-  path) to keep data that is currently referenced from them.
+- `-p <paths>`, `--projects <paths>` - if a single remote or a single [cache is
+  shared] among different projects, this option can be used to specify a list of
+  them (each project is a path) to keep data that is currently referenced from
+  them.
 
 - `-c`, `--cloud` - remove files in remote storage in addition to local cache.
   **This option is dangerous.** The default remote is used unless a specific one
@@ -112,6 +131,9 @@ use the `--remote` (`-r`) option.
 
 - `-v`, `--verbose` - displays detailed tracing information.
 
+[cache is shared]: /doc/user-guide/how-to/share-a-dvc-cache
+[dvc experiments]: /doc/user-guide/experiment-management#experiments
+
 ## Examples
 
 Basic example of cleaning up the <abbr>cache</abbr>:

packages/example/content/docs/command-reference/import.md

@@ -38,10 +38,14 @@ working directory with its original file name e.g. `data.txt` (or to a location
 provided with `--out`). An _import `.dvc` file_ is created in the same location
 e.g. `data.txt.dvc` – similar to using `dvc add` after downloading the data.
 
-(ℹ️) DVC won't push data imported from other DVC repos to
+<admon type="info">
+
+DVC won't push data imported from other DVC repos to
 [remote storage](/doc/command-reference/remote). `dvc pull` will download from
 the original source.
 
+</admon>
+
 The `url` argument specifies the address of the DVC or Git repository containing
 the data source. Both HTTP and SSH protocols are supported (e.g.
 `[user@]server:project.git`). `url` can also be a local file system path
@@ -59,7 +63,7 @@ targets must be found in a `dvc.yaml` or `.dvc` file of the repo.
 a Git server). In such a `.dvc` file, the `deps` field specifies the `url` and
 data `path`, and the `outs` field contains the corresponding local path in the
 <abbr>workspace</abbr>. It records enough metadata about the imported data to
-enable DVC efficiently determining whether the local copy is out of date.
+enable DVC to efficiently determine whether the local copy is out of date.
 
 To actually [version the data](/doc/start/data-and-model-versioning), `git add`
 (and `git commit`) the import `.dvc` file.
@@ -339,7 +343,7 @@ This will result in the following directory structure, which contains a chained
 import and a regular one:
 
 ```
-/repo/d
+/repo/c
 ├── training
 │   ├── data.csv
 │   └── labels

packages/example/content/docs/command-reference/plots/index.md

@@ -216,7 +216,7 @@ Difference in this metric between the current project version and the previous
 commit:
 
 ```dvc
-$ dvc plots diff -d logs.csv HEAD^
+$ dvc plots diff HEAD^ --targets logs.csv
 file:///Users/usr/src/dvc_plots/index.html
 ```
 

packages/example/content/docs/command-reference/repro.md

@@ -9,8 +9,9 @@ correct order.
 ```usage
 usage: dvc repro [-h] [-q | -v] [-f] [-i]
                  [-s] [-p] [-P] [-R] [-m]
-                 [--downstream] [--force-downstream] [--glob]
-                 [--dry] [--no-commit] [--no-run-cache] [--pull]
+                 [--downstream] [--force-downstream]
+                 [--pull] [--dry]
+                 [--glob] [--no-commit] [--no-run-cache]
                  [targets [<target> ...]]
 
 positional arguments:

packages/example/content/docs/install/linux.md

@@ -22,7 +22,7 @@ plan to use, you might need to install optional dependencies: `[s3]`,
 `[gdrive]`, `[gs]`, `[azure]`, `[ssh]`, `[hdfs]`, `[webdav]`, `[oss]`. Use
 `[all]` to include them all.
 
-<details>
+<details id="example-pip-with-support-for-amazon-s3-storage">
 
 ### Example: with support for Amazon S3 storage
 
@@ -53,7 +53,7 @@ Depending on the type of the [remote storage](/doc/command-reference/remote) you
 plan to use, you might need to install optional dependencies: `dvc-s3`,
 `dvc-azure`, `dvc-gdrive`, `dvc-gs`, `dvc-oss`, `dvc-ssh`.
 
-<details>
+<details id="example-conda-with-support-for-amazon-s3-storage">
 
 ### Example: with support for Amazon S3 storage
 
@@ -79,7 +79,7 @@ $ snap install --classic dvc
 
 ## Install from repository
 
-<details>
+<details id="from-repo-on-debian-ubuntu">
 
 ### On Debian/Ubuntu
 
@@ -94,7 +94,7 @@ $ sudo apt install dvc
 
 </details>
 
-<details>
+<details id="from-repo-on-fedora-centos">
 
 ### On Fedora/CentOS
 
@@ -115,7 +115,7 @@ Get the binary package from the big "Download" button on the [home page](/), or
 from the [release page](https://github.com/iterative/dvc/releases/) on GitHub.
 Then install it with the following command.
 
-<details>
+<details id="from-pkg-on-debian-ubuntu">
 
 ### On Debian/Ubuntu
 
@@ -125,7 +125,7 @@ $ sudo apt install ./dvc_0.62.1_amd64.deb
 
 </details>
 
-<details>
+<details id="from-pkg-on-fedora-centos">
 
 ### On Fedora/CentOS
 

packages/example/content/docs/install/macos.md

@@ -43,7 +43,7 @@ plan to use, you might need to install optional dependencies: `[s3]`,
 `[gdrive]`, `[gs]`, `[azure]`, `[ssh]`, `[hdfs]`, `[webdav]`, `[oss]`. Use
 `[all]` to include them all.
 
-<details>
+<details id="example-pip-with-support-for-amazon-s3-storage">
 
 ### Example: with support for Amazon S3 storage
 
@@ -69,7 +69,7 @@ Depending on the type of the [remote storage](/doc/command-reference/remote) you
 plan to use, you might need to install optional dependencies: `dvc-s3`,
 `dvc-azure`, `dvc-gdrive`, `dvc-gs`, `dvc-oss`, `dvc-ssh`.
 
-<details>
+<details id="example-conda-with-support-for-amazon-s3-storage">
 
 ### Example: with support for Amazon S3 storage
 

packages/example/content/docs/install/pre-release.md

@@ -10,9 +10,6 @@ releases, you can install it from our code repository GitHub.
 > (on Python 3.7+) to encapsulate your local environment.
 
 ```dvc
-# Should be installed before DVC. See details below.
-$ pip install gitpython
-
 # Basic version.
 $ pip install git+https://github.com/iterative/dvc
 

packages/example/content/docs/install/windows.md

@@ -35,7 +35,7 @@ Depending on the type of the [remote storage](/doc/command-reference/remote) you
 plan to use, you might need to install optional dependencies: `dvc-s3`,
 `dvc-azure`, `dvc-gdrive`, `dvc-gs`, `dvc-oss`, `dvc-ssh`.
 
-<details>
+<details id="example-conda-with-support-for-amazon-s3-storage">
 
 ### Example: with support for Amazon S3 storage
 
@@ -61,7 +61,7 @@ Depending on the type of the [remote storage](/doc/command-reference/remote) you
 plan to use, you might need to install optional dependencies: `[s3]`, `[azure]`,
 `[gdrive]`, `[gs]`, `[oss]`, `[ssh]`. Use `[all]` to include them all.
 
-<details>
+<details id="example-pip-with-support-for-amazon-s3-storage">
 
 ### Example: with support for Amazon S3 storage