Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

start: Experiments Trail #2574

Merged
merged 90 commits into from
Oct 18, 2021
Merged
Changes from 1 commit
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
a84c2be
experiments draft 1 moved to experiments-trail/
Jul 2, 2021
81c4052
modifications for downloading the data and setting up the experiment …
Jul 2, 2021
e80af13
Fix project installation and commands
iesahin Jul 3, 2021
bb56fb7
added second draft for experiments
Jul 5, 2021
29c3ae0
Restyled by prettier (#2608)
restyled-io[bot] Jul 5, 2021
e12f222
Update version 1 by some command outputs
iesahin Jul 5, 2021
1a646bb
Added naming section and two screenshots to draft 1
iesahin Jul 6, 2021
5c37246
adding separate section for experiment preparation in draft 2
iesahin Jul 6, 2021
5756cfc
fixed link to the project and added some explanations to installation
iesahin Jul 6, 2021
1645b10
some minor edits in draft 1
iesahin Jul 6, 2021
0acdd13
minor edits in pipeline explanation
iesahin Jul 6, 2021
0f43b57
carried over changes in draft-1 and updated the install instructions
iesahin Jul 6, 2021
2f1cd50
merged and simplified the first paragraph and added emojis
iesahin Jul 8, 2021
51aa9da
added links to the sidebar (tentative)
iesahin Jul 8, 2021
9f25143
changes after jorge's review
iesahin Jul 9, 2021
e24d163
carried over changes in alt-1 to alt-2
iesahin Jul 12, 2021
da4d617
added 3rd alternative to experiments trail
iesahin Jul 12, 2021
77c36c2
minor changes
iesahin Jul 12, 2021
e76ff2b
removed alt-1 and alt-2
iesahin Jul 16, 2021
4a6cba7
renamed experiments trail alt-3 and modified the sidebar
iesahin Jul 16, 2021
3b8cbe1
minor changes for experiment commands
iesahin Jul 20, 2021
4c6b54d
updated the experiments and removed some text
iesahin Jul 22, 2021
a2063cc
Updated and simplified initial paragraphs
iesahin Jul 28, 2021
7a6a70b
updated the details section scope to hide all installation info
iesahin Jul 30, 2021
e07584c
moved params diff explanation to hidden section + minor edits
iesahin Jul 30, 2021
cb777d3
replaced some emoji
iesahin Aug 19, 2021
a1e6b6b
removed the "note on experiments names" section
iesahin Aug 19, 2021
c10457f
removed "how dvc updates parameters" section
iesahin Aug 19, 2021
aa30fdf
removed dvc exp apply
iesahin Aug 19, 2021
a8f0741
removed screenshots
iesahin Aug 20, 2021
51afaf5
edits for clarification
iesahin Aug 20, 2021
96406ec
revised the doc per reviews
iesahin Oct 4, 2021
543596e
Update content/docs/start/experiments-trail/experiments.md
iesahin Oct 4, 2021
b4fc360
minor edits
iesahin Oct 4, 2021
cadf03a
some emoji changes
iesahin Oct 4, 2021
9dde3f7
put the video
iesahin Oct 5, 2021
9a56f80
replaced older gs:experiments with the trail doc
iesahin Oct 5, 2021
d1f929a
Removed emoji and moved persisting to details section
iesahin Oct 5, 2021
d085d77
added a sentence for data management
iesahin Oct 5, 2021
797fc1d
added a brief `dvc exp show` after the first run.
iesahin Oct 6, 2021
34c165f
added emoji to detail headers
iesahin Oct 6, 2021
46411bf
added an initial text
iesahin Oct 6, 2021
72cca7c
some suggestions of Ivan applied.
iesahin Oct 8, 2021
dad083d
experiments draft 1 moved to experiments-trail/
Jul 2, 2021
7f768e2
modifications for downloading the data and setting up the experiment …
Jul 2, 2021
761c864
Fix project installation and commands
iesahin Jul 3, 2021
506f0fa
added second draft for experiments
Jul 5, 2021
e7c1127
Restyled by prettier (#2608)
restyled-io[bot] Jul 5, 2021
05fef9e
Update version 1 by some command outputs
iesahin Jul 5, 2021
e9d94c5
Added naming section and two screenshots to draft 1
iesahin Jul 6, 2021
518cb57
adding separate section for experiment preparation in draft 2
iesahin Jul 6, 2021
8e3f186
fixed link to the project and added some explanations to installation
iesahin Jul 6, 2021
14518f9
some minor edits in draft 1
iesahin Jul 6, 2021
ac24222
minor edits in pipeline explanation
iesahin Jul 6, 2021
fcf8aea
carried over changes in draft-1 and updated the install instructions
iesahin Jul 6, 2021
b0d48d3
merged and simplified the first paragraph and added emojis
iesahin Jul 8, 2021
fbf8c5e
added links to the sidebar (tentative)
iesahin Jul 8, 2021
f12d1b8
changes after jorge's review
iesahin Jul 9, 2021
91ee97b
carried over changes in alt-1 to alt-2
iesahin Jul 12, 2021
b7ad284
added 3rd alternative to experiments trail
iesahin Jul 12, 2021
0b97728
minor changes
iesahin Jul 12, 2021
5a925e9
removed alt-1 and alt-2
iesahin Jul 16, 2021
3c6a9b2
renamed experiments trail alt-3 and modified the sidebar
iesahin Jul 16, 2021
8433c8f
minor changes for experiment commands
iesahin Jul 20, 2021
f6b870b
updated the experiments and removed some text
iesahin Jul 22, 2021
b2050b9
Updated and simplified initial paragraphs
iesahin Jul 28, 2021
690b391
updated the details section scope to hide all installation info
iesahin Jul 30, 2021
4be6138
moved params diff explanation to hidden section + minor edits
iesahin Jul 30, 2021
c2e00f5
replaced some emoji
iesahin Aug 19, 2021
f4dae65
removed the "note on experiments names" section
iesahin Aug 19, 2021
77ff0c5
removed "how dvc updates parameters" section
iesahin Aug 19, 2021
397cc53
removed dvc exp apply
iesahin Aug 19, 2021
065fa71
removed screenshots
iesahin Aug 20, 2021
5c2725b
edits for clarification
iesahin Aug 20, 2021
b7d37dc
revised the doc per reviews
iesahin Oct 4, 2021
4e81e46
Update content/docs/start/experiments-trail/experiments.md
iesahin Oct 4, 2021
02cd702
minor edits
iesahin Oct 4, 2021
62edbd8
some emoji changes
iesahin Oct 4, 2021
10e6ec3
put the video
iesahin Oct 5, 2021
bd4b6ad
replaced older gs:experiments with the trail doc
iesahin Oct 5, 2021
8dfe709
Removed emoji and moved persisting to details section
iesahin Oct 5, 2021
56ccd4c
added a sentence for data management
iesahin Oct 5, 2021
2a8b42e
added a brief `dvc exp show` after the first run.
iesahin Oct 6, 2021
769c2e6
added emoji to detail headers
iesahin Oct 6, 2021
c7ddcf8
added an initial text
iesahin Oct 6, 2021
915c61a
some suggestions of Ivan applied.
iesahin Oct 8, 2021
ea94da6
modifications after Dave's review
iesahin Oct 15, 2021
2b9ebd9
Merge branch 'iesahin/new-gs-experiments' of github.com:iterative/dvc…
iesahin Oct 15, 2021
ec3d132
merged
iesahin Oct 15, 2021
d3cae7a
revised after Dave's review.
iesahin Oct 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
some suggestions of Ivan applied.
iesahin committed Oct 8, 2021
commit 72cca7c4274e264e71d61bc90f32f19e1ec8b2e3
29 changes: 11 additions & 18 deletions content/docs/start/experiments.md
Original file line number Diff line number Diff line change
@@ -54,14 +54,6 @@ experiments.

</details>

## Running the experiment with default parameters

The purpose of `dvc exp` subcommands is to let you run, capture, compare the
machine learning experiments at once as you iterate on your code, data, and
hyperparameters. The artifacts like models, metrics produced by each experiment
are tracked by DVC and the associated parameters and metrics can be committed to
Git as text files.

Running the experiment with the default project settings requires only the
command:

@@ -80,6 +72,12 @@ This experiment is then associated with the values found in parameters file
(`params.yaml`), and other dependencies (`data/images/`) with these produced
metrics.

The purpose of `dvc exp` family of commands is to let you run, capture, compare
iesahin marked this conversation as resolved.
Show resolved Hide resolved
the machine learning experiments at once as you iterate on your code, data,
iesahin marked this conversation as resolved.
Show resolved Hide resolved
models, and hyperparameters. The artifacts like models, metrics produced by each
iesahin marked this conversation as resolved.
Show resolved Hide resolved
experiment are tracked by DVC and the associated parameters and metrics can be
iesahin marked this conversation as resolved.
Show resolved Hide resolved
committed to Git as text files.

You can review the experiment results with `dvc exp show` and see these metrics
and results in a nicely formatted table:

@@ -98,7 +96,10 @@ $ dvc exp show
```

The `workspace` row in the table shows the results of the most recent experiment
that's available in the <bbr>workspace</abbr>.
that's available in the <bbr>workspace</abbr>. The table also shows each
iesahin marked this conversation as resolved.
Show resolved Hide resolved
experiment in a separate row, along with the Git commit IDs they are attached
to. We can see that the experiment we run has a name `exp-6dccf` and run from
iesahin marked this conversation as resolved.
Show resolved Hide resolved
the commit ID `7317bc6`.

<details>

@@ -119,8 +120,6 @@ experiments easily.

</details>

## Running the experiment by setting parameters

Now let's do some more experimentation.

DVC allows to update the parameters defined in the pipeline without modifying
@@ -165,7 +164,7 @@ $ dvc exp run --run-all --jobs 2

</details>

## Comparing experiments
## Comparing and persisting experiments

The experiments are run several times with different parameters. We use
`dvc exp show` to compare all of these experiments. This command presents the
@@ -216,10 +215,6 @@ $ dvc exp show --no-timestamp \
└─────────────────────────┴────────┴──────────────────┘
```

<details>

### ✅ Persisting experiments

After selecting an experiment from the table, it's possible to apply the
experiment results to your workspace, or you can create a Git branch that
contains the experiment with all its related files.
@@ -235,8 +230,6 @@ To switch to the new branch run:
You can then checkout and continue working from this branch, or merge the branch
your `main` branch with usual Git commands.
iesahin marked this conversation as resolved.
Show resolved Hide resolved

</details>

## Go Further

There are many other features of `dvc exp`, like cleaning up the unused