Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref: main exp workflow (run,show,diff,apply) #2200

Merged
merged 40 commits into from
Mar 3, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
765829e
ref: add run,show,apply cmds
jorgeorpinel Feb 16, 2021
d0678f4
Merge branch 'ref/experiments' into ref/exp/main
jorgeorpinel Feb 16, 2021
a250f3a
Merge branch 'ref/experiments' into ref/exp/main
jorgeorpinel Feb 16, 2021
d154a43
Merge branch 'ref/experiments' into ref/exp/main
jorgeorpinel Feb 17, 2021
57c3aee
Merge branch 'ref/experiments' into ref/exp/main
jorgeorpinel Feb 17, 2021
1ba51c3
Merge branch 'ref/experiments' into ref/exp/main
jorgeorpinel Feb 17, 2021
4cffc66
ref: write Description and dump Options for exp run
jorgeorpinel Feb 18, 2021
de5fd76
Merge branch 'ref/experiments' into ref/exp/main
jorgeorpinel Feb 21, 2021
e902a4d
Merge branch 'ref/experiments' into ref/exp/main
jorgeorpinel Feb 21, 2021
af329a7
Merge branch 'ref/experiments' into ref/exp/main
jorgeorpinel Feb 21, 2021
cdaa44b
Merge branch 'master' into ref/exp/main
jorgeorpinel Feb 23, 2021
d8c2c1c
ref: updates to existing api and exp index, and new exp cmds (inc. diff)
jorgeorpinel Feb 23, 2021
efa52e4
ref: exm run impros
jorgeorpinel Feb 23, 2021
a7f62f3
ref: mention exp names in exp run/diff
jorgeorpinel Feb 23, 2021
8891422
ref: dump exp diff options
jorgeorpinel Feb 23, 2021
7b451d6
ref: dump exp show options
jorgeorpinel Feb 23, 2021
125f92d
ref: add exp apply options
jorgeorpinel Feb 23, 2021
efe6c53
ref: update exp diff
jorgeorpinel Feb 23, 2021
23fa604
ref: doc checkpoints in exp run and impro make_checkpoint
jorgeorpinel Feb 23, 2021
c5723b7
exp: updates to refs and guides per DAve's feedback
jorgeorpinel Feb 23, 2021
971a04a
ref: complete exp run ref details and options
jorgeorpinel Feb 24, 2021
ad53ba2
exps: clarifications in run ref; add .dvc/tmp/expn to internal files …
jorgeorpinel Feb 25, 2021
3d6d16e
ref: review of exp diff options
jorgeorpinel Feb 25, 2021
53f02ab
ref: explain make_checkpoint algo
jorgeorpinel Feb 25, 2021
e96e251
ref: mention manual signal file process for checkpoints in non-Python…
jorgeorpinel Feb 25, 2021
507427f
ref: correct exp diff default behavior
jorgeorpinel Feb 25, 2021
f55380e
ref: review exp show option descs
jorgeorpinel Feb 25, 2021
7bd090f
ref: add exp diff to nav! and intro desc examples for exp show and diff
jorgeorpinel Feb 25, 2021
6a619b1
ref: copy edits to exp apply
jorgeorpinel Feb 25, 2021
0fb0fcb
Merge branch 'master' into ref/exp/main
jorgeorpinel Feb 25, 2021
7456150
Merge branch 'master' into ref/exp/main
jorgeorpinel Feb 27, 2021
cb325bb
exp: clarify about checkpoint outputs
jorgeorpinel Feb 27, 2021
6e5f486
ref: on exp show pagination and sorting
jorgeorpinel Feb 27, 2021
2497d55
guide: Exps workflow
jorgeorpinel Feb 27, 2021
a16d83e
ref: finish exp apply Desc
jorgeorpinel Feb 27, 2021
9063545
ref: updates to exp show Desc
jorgeorpinel Feb 27, 2021
2e26385
ref: more info about sorting exp show
jorgeorpinel Feb 27, 2021
0fb378c
exp: warn about repro and checkpoint outs + review "checkpoint" term
jorgeorpinel Feb 27, 2021
18fa527
ref: exp show hide/show vs sort
jorgeorpinel Feb 27, 2021
c4b0a01
Merge branch 'master' into ref/exp/main
jorgeorpinel Feb 28, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
guide: Exps workflow
  • Loading branch information
jorgeorpinel committed Feb 27, 2021
commit 2497d55f85b2011737dab48e612de3ff318fbe73
22 changes: 5 additions & 17 deletions content/docs/command-reference/exp/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

⚠️ This feature is only available in DVC 2.0 ⚠️

A set of commands to generate and manage <abbr>experiment</abbr>:
A set of commands to generate and manage <abbr>experiments</abbr>:
[run](/doc/command-reference/exp/run), [show](/doc/command-reference/exp/show),
[diff](/doc/command-reference/exp/diff),
[apply](/doc/command-reference/exp/apply),
Expand Down Expand Up @@ -38,26 +38,14 @@ positional arguments:

## Description

`dvc exp` subcommands provide specialized ways to create and manage data science
experiments. The basic workflow goes like this:

- Modify stage <abbr>parameters</abbr> or other dependencies (e.g. input data,
source code) of committed stages.
- Use `dvc exp run` (instead of `repro`) to execute the pipeline. The results
are reflected in your <abbr>workspace</abbr>, and tracked automatically.
- Use [metrics](/doc/command-reference/metrics) in your pipeline to identify the
best experiment(s).
- Visualize and compare experiments with `dvc exp show` or `dvc exp diff`.
Repeat.
- Use `dvc exp apply` to roll back to the best one.
- Make the selected experiment persistent by committing its results to Git. This
cleans the slate so you can repeat the process.
`dvc exp` subcommands provide specialized ways to create and manage data
science/ machine learning experiments.

📖 See [Experiment Management](/doc/user-guide/experiment-management) for more
info.

> Note that DVC assumes that <abbr>experiments</abbr> are deterministic (see
> **Avoiding unexpected behavior** in `dvc run`).
> ⚠️ Note that DVC assumes that experiments are deterministic (see **Avoiding
> unexpected behavior** in `dvc stage add`).

## Options

Expand Down
10 changes: 5 additions & 5 deletions content/docs/user-guide/experiment-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ models. On the other end, [metrics](/doc/command-reference/metrics) (and
[plots](/doc/command-reference/plots)) let you define, visualize, and compare
meaningful measures for the experimental results.

> 👨‍💻 See [Get Started: Experiments](/doc/start/experiments) for a hands-on
> introduction to DVC experiments.

## Experiments

⚠️ This feature is only available in DVC 2.0 ⚠️
Expand All @@ -43,8 +46,8 @@ roll back to the baseline. The basic workflow goes like this:
are reflected in your <abbr>workspace</abbr>, and tracked automatically.
- Use [metrics](/doc/command-reference/metrics) to identify the best
experiment(s).
- Visualize and compare experiments with `dvc exp show` or `dvc exp diff`.
Repeat.
- Visualize, compare experiments with `dvc exp show` or `dvc exp diff`. Repeat
🔄
- Use `dvc exp apply` to roll back to the best one.
- Make the selected experiment persistent by committing its results to Git. This
cleans the slate so you can repeat the process.
Expand Down Expand Up @@ -74,9 +77,6 @@ version. The right <abbr>outputs</abbr> (including
[metrics](/doc/command-reference/metrics)) should also be present, or available
via `dvc checkout`.

> 👨‍💻 See [Get Started: Experiments](/doc/start/experiments) for a hands-on
> introduction to regular experiments.

### Organization patterns

DVC takes care of arranging `dvc exp` experiments and the data
Expand Down