DVCLive is a Python library for logging machine learning metrics and other
metadata in simple file formats, which is fully compatible with DVC.
Install dvclive
Initialize DVC Repository
$ git init $ dvc init $ git commit -m "DVC init"
Example code
Copy the snippet below as a basic example of the API usage:
# train.py import random import sys from dvclive import Live with Live(save_dvc_exp=True) as live: epochs = int(sys.argv[1]) live.log_param("epochs", epochs) for epoch in range(epochs): live.log_metric("train/accuracy", epoch + random.random()) live.log_metric("train/loss", epochs - epoch - random.random()) live.log_metric("val/accuracy",epoch + random.random() ) live.log_metric("val/loss", epochs - epoch - random.random()) live.next_step()
See Integrations for examples using
DVCLive alongside different ML Frameworks.
Running
Run couple of times passing different values:
$ python train.py 5 $ python train.py 5 $ python train.py 7
Comparing
DVCLive outputs can be rendered in different ways:
DVC CLI
You can use dvc exp show and
dvc plots to compare and
visualize metrics, parameters and plots across experiments:
─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Experiment Created train.accuracy train.loss val.accuracy val.loss step epochs
─────────────────────────────────────────────────────────────────────────────────────────────────────────────
workspace - 6.0109 0.23311 6.062 0.24321 6 7
master 08:50 PM - - - - - -
├── 4475845 [aulic-chiv] 08:56 PM 6.0109 0.23311 6.062 0.24321 6 7
├── 7d4cef7 [yarer-tods] 08:56 PM 4.8551 0.82012 4.5555 0.033533 4 5
└── d503f8e [curst-chad] 08:56 PM 4.9768 0.070585 4.0773 0.46639 4 5
─────────────────────────────────────────────────────────────────────────────────────────────────────────────
$ dvc plots diff $(dvc exp list --names-only) --open
DVC Extension for VS Code
Inside the
DVC Extension for VS Code,
you can compare and visualize results using the
Experiments
and
Plots
views:
While experiments are running, live updates will be displayed in both views.
DVC Studio
If you push the results to DVC Studio, you can
compare experiments against the entire repo history:
You can enable
Studio Live Experiments
to see live updates while experiments are running.
DVCLive is an ML Logger, similar to:
The main difference with those ML Loggers is that DVCLive does not
require any additional services or servers to run.
Logged metrics, parameters, and plots are stored as plain text files that can be
versioned by tools like Git or tracked as pointers to files in DVC storage.
You can then use different options to visualize the metrics,
parameters, and plots across experiments.
Contributions are very welcome. To learn more, see the
Contributor Guide.
Distributed under the terms of the
Apache 2.0 license, dvclive is
free and open source software.
