Skip to content

Commit

Permalink
cmd ref: add list command (#1021)
Browse files Browse the repository at this point in the history
* api: add list command

* cmd ref: reqrite description (examples pending review) and link from get/import

* api ref: list copy edits

* cmd ref: target->path in list
per #1021 (comment)

Co-authored-by: Jorge Orpinel <[email protected]>
  • Loading branch information
gurobokum and jorgeorpinel authored Mar 14, 2020
1 parent 01ed81c commit ae1761b
Show file tree
Hide file tree
Showing 4 changed files with 153 additions and 0 deletions.
3 changes: 3 additions & 0 deletions public/static/docs/command-reference/get.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ directory. (Analogous to `wget`, but for repos.)
> files (does not create a DVC-file). For that reason, this command doesn't
> require an existing DVC project to run in.
> See `dvc list` for a way to browse repository contents to find files or
> directories to download.
The `url` argument specifies the address of the DVC or Git repository containing
the data source. Both HTTP and SSH protocols are supported for online repos
(e.g. `[user@]server:project.git`). `url` can also be a local file system path
Expand Down
3 changes: 3 additions & 0 deletions public/static/docs/command-reference/import.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ updating the import later, if it has changed in its data source. (See
> Note that `dvc get` corresponds to the first step this command performs (just
> download the data).
> See `dvc list` for a way to browse repository contents to find files or
> directories to import.
The `url` argument specifies the address of the DVC or Git repository containing
the data source. Both HTTP and SSH protocols are supported for online repos
(e.g. `[user@]server:project.git`). `url` can also be a local file system path
Expand Down
143 changes: 143 additions & 0 deletions public/static/docs/command-reference/list.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# list

List repository contents, including files and directories tracked by DVC
(<abbr>data artifacts</abbr>) and by Git.

## Synopsis

```usage
usage: dvc list [-h] [-q | -v] [-R] [--outs-only] [--rev [REV]]
url [path]
positional arguments:
url Location of DVC or Git repository to list from
path Path to a file or directory within the repository
```

## Description

Lists files and directories in the root of a <abbr>repository</abbr>, including
<abbr>data artifacts</abbr> tracked by DVC (e.g. data, models), and Git-tracked
files (e.g. source code). To list recursively, use the `-R` option. The list is
sorted alphabetically.

This command is especially useful to browse a public repo in order to find the
exact file or directory names to `dvc import` or `dvc get`.

Note that this command doesn't require an existing DVC project to run in. Also,
it doesn't support listing <abbr>DVC projects</abbr> that are not tracked by Git
(see the `--no-scm` option of `dvc init`).

The `url` argument specifies the address of the DVC or Git repository to list.
Both HTTP and SSH protocols are supported for online repos (e.g.
`[user@]server:project.git`). `url` can also be a local file system path to an
"offline" repo.

The `path` argument of this command is used to specify a path within the source
repository at `url`. If the path is a file and it's found in the repo, it will
be printed back as a confirmation of its existence. If it's a Git-tracked
directory, files and directories directly under it will be listed (use option
`-R` to list recursively).

> Listing the contents of DVC-tracked directories is not supported at the time.
## Options

- `-R`, `--recursive` - recursively prints the repository contents. (It can be
limited to a specific Git-tracked directory by supplying a `path` argument.)

- `--outs-only` - show only DVC-tracked files and directories
(<abbr>outputs</abbr>).

- `--rev` - commit hash, branch or tag name, etc. (any
[Git revision](https://git-scm.com/docs/revisions)) of the repository to list
content for. The latest commit in `master` (tip of the default branch) is used
by default when this option is not specified.

- `-h`, `--help` - prints the usage/help message, and exit.

- `-q`, `--quiet` - do not write anything to standard output. Exit with 0 if no
problems arise, otherwise 1.

- `-v`, `--verbose` - displays detailed tracing information. when this option is
not specified.

## Example: List remote git repo

We can use the command for getting information about remote repository with all
files, directories and <abbr>data artifacts</abbr>.

```dvc
$ dvc list -R https://github.com/iterative/dataset-registry
.gitignore
README.md
get-started/.gitignore
get-started/data.xml
get-started/data.xml.dvc
images/.gitignore
images/dvc-logo-outlines.png
images/dvc-logo-outlines.png.dvc
images/owl_sticker.png
images/owl_sticker.png.dvc
images/owl_sticker.svg
...
```

Or

```dvc
$ dvc list https://github.com/iterative/dataset-registry
.gitignore
README.md
get-started
images
tutorial
use-cases
```

for getting flat information about the repo

## Example: List the repo with the rev

Another useful case is checking the files for the **specific revision**

```dvc
$ dvc list -R --rev 7476a858f6200864b5755863c729bff41d0fb045 \
https://github.com/iterative/dataset-registry
.gitignore
README.md
get-started/.gitignore
get-started/data.xml
get-started/data.xml.dvc
tutorial/nlp/.gitignore
tutorial/nlp/Posts.xml.zip
tutorial/nlp/Posts.xml.zip.dvc
tutorial/nlp/pipeline.zip
tutorial/nlp/pipeline.zip.dvc
tutorial/ver/.gitignore
tutorial/ver/data.zip
tutorial/ver/data.zip.dvc
tutorial/ver/new-labels.zip
```

## Example: Check the path

Before trying to get or import some <abbr>data artifacts</abbr> with `dvc get`
or `dvc import` we can check their existence with

```dvc
$ dvc list --outs-only \
https://github.com/iterative/dataset-registry \
tutorial/nlp/pipeline.zip
```

Or everything under the prefix

```dvc
$ dvc list -R --outs-only \
https://github.com/iterative/dataset-registry \
tutorial
```
4 changes: 4 additions & 0 deletions public/static/docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -241,6 +241,10 @@
"label": "install",
"slug": "install"
},
{
"label": "list",
"slug": "list"
},
{
"label": "lock",
"slug": "lock"
Expand Down

0 comments on commit ae1761b

Please sign in to comment.