Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regular updates (March 18) #1072

Merged
merged 15 commits into from
Mar 26, 2020
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions content/docs/command-reference/gc.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ files if no scope options are provided. It means it's user's responsibility to
explicitly provide the right set of options to specify what data is still needed
(so that DVC can figure out what fils can be safely deleted).

One of the scope options, `--workspace`, `--all-branches`, `--all-tags`,
`--all-commits`, or any combination of them must be provided. Each of them
corresponds to the current workspace _and_ a set of commits to analyze what
files, directories and what versions are still needed and should be kept (by
analyzing DVC-files in those commits).
One of the scope options (`--workspace`, `--all-branches`, `--all-tags`,
`--all-commits`) or a combination of them must be provided. Each of them
corresponds to keeping the data for the current workspace, and for a certain set
of commits (determined by reading the DVC-files in them). See the
[Options](#options) section for more details.

Unless the `--cloud` option is used, `dvc gc` does not remove data files from
any remote. This means that any files collected from the local cache can be
Expand Down Expand Up @@ -61,10 +61,11 @@ The default remote is cleaned (see `dvc config core.remote`) unless the
commits as well as the workspace (implies `-w`). Useful for keeping all the
data used in the entire existing commit history of the project.

One of the use cases for this option is to safely delete all temporary data
DVC cached when `dvc run` and/or `dvc repro` were run without committing
changes to DVC-files (thus potentially caching data that is not referenced
from workspace or Git commits).
A use case for this option is to safely delete all temporary data `dvc run`
and/or `dvc repro` cache when used without committing changes (see the `-O` or
`-M`, and `--no-commit` options in those commands). In that scenario, data
that is never referenced from the workspace or from any Git commit can still
be stored in the project's cache).

- `-p <paths>`, `--projects <paths>` - if a single remote or a single cache is
shared among different projects (e.g. a configuration like the one described
Expand Down
13 changes: 7 additions & 6 deletions content/docs/command-reference/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,11 @@ directory:
To disable them, you need to **remove** or **edit** those files (i.e.
`rm .git/hooks/post-checkout`, `vim .git/hooks/pre-commit`).

## Using Pre-Commit tool
## Using the Pre-commit tool

DVC provides support for [pre-commit](https://pre-commit.com/) users. To adjust
`.pre-commit-config.yaml` you could either use
`dvc install --use-pre-commit-tool`, or add these entries by hand:
DVC provides support to manage Git hooks with
[pre-commit](https://pre-commit.com/). To adjust `.pre-commit-config.yaml`, you
can either use `dvc install --use-pre-commit-tool` or add these entries by hand:

```yaml
repos:
Expand All @@ -103,8 +103,9 @@ repos:

## Options

- `--use-pre-commit-tool` - install pre-commit/pre-push/post-checkout hooks into
the [pre-commit](https://pre-commit.com/) config (`.pre-commit-config.yaml`).
- `--use-pre-commit-tool` - installs pre-commit, pre-push, post-checkout Git
hooks into the [pre-commit](https://pre-commit.com/) config file
(`.pre-commit-config.yaml`).

- `-h`, `--help` - prints the usage/help message, and exit.

Expand Down
3 changes: 1 addition & 2 deletions content/docs/command-reference/metrics/add.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# metrics add

Mark <abbr>output</abbr> file as a
[project metric](/doc/command-reference/metrics).
Mark a DVC-tracked file as a [project metric](/doc/command-reference/metrics).

## Synopsis

Expand Down
6 changes: 3 additions & 3 deletions content/docs/command-reference/metrics/remove.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# metrics remove

Stop tracking a [project metric](/doc/command-reference/metrics): Keeps file
located at `path` as an <abbr>output</abbr>, but removes its metric mark in the
DVC-file.
Remove metric mark on a DVC-tracked file: Keeps file located at `path` as an
<abbr>output</abbr>, but removes its mark as a
[project metric](/doc/command-reference/metrics) in the DVC-file.

## Synopsis

Expand Down
2 changes: 1 addition & 1 deletion content/docs/command-reference/remove.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# remove

Remove data files or directories tracked by DVC.
Remove DVC-tracked files or directories from the <abbr>workspace</abbr>.

## Synopsis

Expand Down
9 changes: 5 additions & 4 deletions content/docs/command-reference/root.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# root

Return the relative path to the <abbr>DVC project</abbr>.
Return the relative path to the root of the <abbr>DVC project</abbr>.

## Synopsis

Expand All @@ -11,9 +11,10 @@ usage: dvc root [-h] [-q | -v]
## Description

While in sub-directories of the project, sometimes developers may want to refer
some file belonging to another directory. This command returns relative path to
the project root from the current working directory. So this command can be used
to build a path to a dependency file, command, or output.
some file belonging to another directory. This command returns the path to the
root directory of the current <abbr>DVC project</abbr>, relative to the current
working directory. This command can be used to build a path to a dependency
file, command, or output.

## Options

Expand Down
2 changes: 1 addition & 1 deletion content/docs/install/linux.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ plan to use, you might need to install optional dependencies: `[s3]`, `[azure]`,
### Example: How to install DVC with support for Amazon S3 storage

```dvc
$ pip install 'dvc[s3]'
$ pip install "dvc[s3]"
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
```

In this case it installs `boto3` library as well, besides DVC.
Expand Down
2 changes: 1 addition & 1 deletion content/docs/install/macos.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ plan to use, you might need to install optional dependencies: `[s3]`, `[azure]`,
### Example: How to install DVC with support for Amazon S3 storage

```dvc
$ pip install 'dvc[s3]'
$ pip install "dvc[s3]"
```

In this case it installs `boto3` library as well, besides DVC.
Expand Down
9 changes: 4 additions & 5 deletions content/docs/install/windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ on GitHub.

You can use `conda` from
[Anaconda Prompt](https://docs.anaconda.com/anaconda/user-guide/getting-started/#open-prompt-win),
which is a POSIX-like command line terminal in Windows.
a POSIX-like command line terminal in Windows.

```dvc
$ conda install -c conda-forge dvc
Expand All @@ -36,9 +36,8 @@ $ conda install -c conda-forge dvc

## Install with pip

You can install with `pip` from a command line terminal like
[Git Bash](https://gitforwindows.org/#bash) or
[WSL](https://blogs.windows.com/windowsdeveloper/2016/03/30/run-bash-on-ubuntu-on-windows/).
From Command Prompt or other
[recommended consoles](/doc/user-guide/running-dvc-on-windows):

```dvc
$ pip install dvc
Expand All @@ -53,7 +52,7 @@ plan to use, you might need to install optional dependencies: `[s3]`, `[azure]`,
### Example: How to install DVC with support for Amazon S3 storage

```dvc
$ pip install 'dvc[s3]'
$ pip install "dvc[s3]"
```

In this case it installs `boto3` library as well, besides DVC.
Expand Down
4 changes: 2 additions & 2 deletions content/docs/user-guide/contributing/docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ $ git clone [email protected]:<username>/dvc.org.git
$ cd dvc.org
```

Make sure you have the latest version of [Node.js](https://nodejs.org/en/), and
install [Yarn](https://yarnpkg.com/):
Make sure you have a **recent** version of [Node.js](https://nodejs.org/en/),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does "recent" mean?

Copy link
Contributor Author

@jorgeorpinel jorgeorpinel Mar 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I'm not sure what's the oldest version this works on, probably its safe to recommend 10+ cc @iAdramelk ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its easier to revert to "latest" but will always work? We can't guarantee it, maybe we should get specific. But up to you, no strong opinion from my part.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I would just specify explicitly the versions we support

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is info. I don't currently know exactly. I know the std. node provided by apt-get (Ubuntu) is really old so I had to install like this. @iAdramelk could you chime in here please?

Copy link
Member

@shcheklein shcheklein Mar 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have engines specified as

  "engines": {
    "node": "^12.0.0"
  },

in blog (which will be part of the dvc.org soon). @fabiosantoscode and @iAdramelk should have more context - ping them in Slack if needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pinged Fabian but he's probably offline rn. I'm using 12 for now so we can merge this and I can update with whatever they say in another PR later. Going to focus on other doc PRs for now.

and install [Yarn](https://yarnpkg.com/):

```dvc
$ npm install -g yarn
Expand Down
47 changes: 28 additions & 19 deletions content/docs/user-guide/running-dvc-on-windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,31 @@ involving system performance. Some, for example, have to do with NTFS file
system characteristics and Windows built-in security mechanisms. Below are some
workarounds that can help avoid these potential problems:

## POSIX-like command line shell

Many of the DVC commands have POSIX-style options that are given with a double
dash `--`. This isn't supported by the simple Windows command prompt `cmd`.

Common Windows terminal alternatives are
[Git Bash](https://gitforwindows.org/#bash) or
[Anaconda Prompt](https://docs.anaconda.com/anaconda/user-guide/getting-started/#open-prompt-win)
– but they may not support all the POSIX features (e.g. `\` line continuation).

💡 We recommend the full [Cmder](https://cmder.net/) console emulator (which
already includes _Git for Windows_).

Its also possible to enjoy a full Linux terminal experience with the
[WSL](https://blogs.windows.com/windowsdeveloper/2016/03/30/run-bash-on-ubuntu-on-windows/)
– but it may not be possible to access GPUs from this subsystem.
## POSIX-like command line experience

The regular Command Prompt (`cmd`) in Windows will most likely not help you use
DVC effectively, or follow the examples in our docs. Please avoid it. There's no
perfect solution, bu here are some ideas:

- The full [Cmder](https://cmder.net/) console emulator combines several useful
tools like [ConEmu](https://conemu.github.io/), and
[Git for Windows](https://gitforwindows.org/)\* (Git Bash) among other
[shell options](https://github.com/cmderdev/cmder/blob/master/README.md#access-to-multiple-shells-in-one-window-using-tabs).
- [Anaconda Prompt](https://docs.anaconda.com/anaconda/user-guide/getting-started/#open-prompt-win)
is another recommendation, but it may not support all the desired CLI features
(e.g. `\` line continuation).
- Consider enabling and using
[WSL](https://blogs.windows.com/windowsdeveloper/2016/03/30/run-bash-on-ubuntu-on-windows/)
([Windows Terminal](https://devblogs.microsoft.com/commandline/) also
recommended). But it may be slow and unable to access GPUs, et al.\*
- Install an actual Linux distro (e.g. Ubuntu) on a virtual machine, or in a HD
partition (dual boot).

> \* Please consult this [solution](https://stackoverflow.com/a/1257613/761963)
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
> if you have strange file permission issues with Git for Windows. Also, Running
> Node apps on WSL may produce
> [`EACCESS`](https://github.com/microsoft/WSL/issues/3395) file permission
> issues.

## Disable short-file name generation

Expand Down Expand Up @@ -65,6 +74,6 @@ via [Chocolatey](https://chocolatey.org/) (please install the tool first):
$ choco install less
```

`less` can be installed in other ways, just make sure it's available in
`cmd`/PowerShell, where you run `dvc`. (This usually means adding the directory
where `less` is installed to the `PATH` environment variable.)
`less` can be installed in other ways, just make sure it's available in the
command line environment where you run `dvc`. (This usually means adding the
directory where `less` is installed to the `PATH` environment variable.)