zincware/ZnTrack

View on GitHub
examples/docs/04_metrics_and_plots.ipynb

Summary

Maintainability
Test Coverage
{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a10ab826-497f-41a1-bf34-3c62c0899bf6",
   "metadata": {},
   "source": [
    "# DVC Options - Metrics and Plots\n",
    "## The DVC way\n",
    "\n",
    "In the following part we will look into metrics and plots from ZnTrack Nodes.\n",
    "All `dvc run` options listed [here](https://dvc.org/doc/command-reference/run#options) can be used via `dvc.<option>`.\n",
    "With the exception of params, which is handled automatically.\n",
    "All these options take either `str` or `pathlib.Path` directed to the file the content should be stored in.\n",
    "As shown before, `dvc.deps` can also take another `Node` as an argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "cdd2fd78-67ad-4b06-9320-053c3268d4da",
   "metadata": {},
   "outputs": [],
   "source": [
    "import zntrack\n",
    "from pathlib import Path\n",
    "import json\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9b7facd2-0284-44fd-a3d4-85edbcfbc78c",
   "metadata": {},
   "outputs": [],
   "source": [
    "zntrack.config.nb_name = \"04_metrics_and_plots.ipynb\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cccf0e17-79f1-47c4-ba2c-cdfd08842b8b",
   "metadata": {
    "nbsphinx": "hidden",
    "tags": []
   },
   "outputs": [],
   "source": [
    "from zntrack.utils import cwd_temp_dir\n",
    "\n",
    "temp_dir = cwd_temp_dir()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8157c334-44cd-42e7-b5de-150327a32389",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initialized empty Git repository in /tmp/tmpq5kwayap/.git/\n",
      "Initialized DVC repository.\n",
      "\n",
      "You can now commit the changes to git.\n",
      "\n",
      "+---------------------------------------------------------------------+\n",
      "|                                                                     |\n",
      "|        DVC has enabled anonymous aggregate usage analytics.         |\n",
      "|     Read the analytics documentation (and how to opt-out) here:     |\n",
      "|             <https://dvc.org/doc/user-guide/analytics>              |\n",
      "|                                                                     |\n",
      "+---------------------------------------------------------------------+\n",
      "\n",
      "What's next?\n",
      "------------\n",
      "- Check out the documentation: <https://dvc.org/doc>\n",
      "- Get help and share ideas: <https://dvc.org/chat>\n",
      "- Star us on GitHub: <https://github.com/iterative/dvc>\n"
     ]
    }
   ],
   "source": [
    "!git init\n",
    "\n",
    "!dvc init"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ad172d0-25f9-44f3-a00c-ee50f347ebc1",
   "metadata": {},
   "source": [
    "In the following we define a simple Node that produces a metric and a plot output using `json` and `pandas`.\n",
    "We will queue multiple experiments with different outputs and then compare them afterwards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "de05180e-b3a8-4ea3-a0b6-592f0467b844",
   "metadata": {},
   "outputs": [],
   "source": [
    "class MetricAndPlot(zntrack.Node):\n",
    "    my_metric: Path = zntrack.dvc.metrics(Path(\"my_metric.json\"))\n",
    "    my_plots: Path = zntrack.dvc.plots(\"my_plots.csv\")\n",
    "    pre_factor = zntrack.zn.params(1.0)\n",
    "\n",
    "    def run(self):\n",
    "        self.my_metric.write_text(\n",
    "            json.dumps(\n",
    "                {\"metric_1\": 17 * self.pre_factor, \"metric_2\": 42 * self.pre_factor}\n",
    "            )\n",
    "        )\n",
    "\n",
    "        x_data = np.linspace(0, 1.0 * self.pre_factor, 1000)\n",
    "        y_data = np.exp(x_data)\n",
    "        df = pd.DataFrame({\"y\": y_data, \"x\": x_data}).set_index(\"x\")\n",
    "\n",
    "        df.to_csv(self.my_plots)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "2f43059d-71e9-4b7c-bf9d-67869b05d2f1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Running DVC command: 'stage add --name MetricAndPlot --force ...'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating 'dvc.yaml'\n",
      "Adding stage 'MetricAndPlot' in 'dvc.yaml'\n",
      "\n",
      "To track the changes with git, run:\n",
      "\n",
      "\tgit add .gitignore dvc.yaml\n",
      "\n",
      "To enable auto staging, run:\n",
      "\n",
      "\tdvc config core.autostage true\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Jupyter support is an experimental feature! Please save your notebook before running this command!\n",
      "Submit issues to https://github.com/zincware/ZnTrack.\n",
      "[NbConvertApp] Converting notebook 04_metrics_and_plots.ipynb to script\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running stage 'MetricAndPlot':\n",
      "> zntrack run src.MetricAndPlot.MetricAndPlot --name MetricAndPlot\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[NbConvertApp] Writing 3574 bytes to 04_metrics_and_plots.py\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generating lock file 'dvc.lock'\n",
      "Updating lock file 'dvc.lock'\n",
      "\n",
      "To track the changes with git, run:\n",
      "\n",
      "\tgit add dvc.lock\n",
      "\n",
      "To enable auto staging, run:\n",
      "\n",
      "\tdvc config core.autostage true\n",
      "Use `dvc push` to send your updates to remote storage.\n",
      "[main (root-commit) 57992f7] First Run\n",
      " 11 files changed, 778 insertions(+)\n",
      " create mode 100644 .dvc/.gitignore\n",
      " create mode 100644 .dvc/config\n",
      " create mode 100644 .dvcignore\n",
      " create mode 100644 .gitignore\n",
      " create mode 100644 04_metrics_and_plots.ipynb\n",
      " create mode 100644 dvc.lock\n",
      " create mode 100644 dvc.yaml\n",
      " create mode 100644 params.yaml\n",
      " create mode 100644 src/MetricAndPlot.py\n",
      " create mode 100644 src/__pycache__/MetricAndPlot.cpython-310.pyc\n",
      " create mode 100644 zntrack.json\n"
     ]
    }
   ],
   "source": [
    "with zntrack.Project() as project:\n",
    "    node = MetricAndPlot()\n",
    "\n",
    "project.run()\n",
    "!git add .\n",
    "!git commit -m \"First Run\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "e433e5f1-8908-44ec-afc0-9217b42faf1a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[NbConvertApp] Converting notebook 04_metrics_and_plots.ipynb to script\n",
      "[NbConvertApp] Writing 3574 bytes to 04_metrics_and_plots.py\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Queued experiment 'factor_2' for future execution.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[NbConvertApp] Converting notebook 04_metrics_and_plots.ipynb to script\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Queued experiment 'factor_3' for future execution.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[NbConvertApp] Writing 3574 bytes to 04_metrics_and_plots.py\n",
      "[NbConvertApp] Converting notebook 04_metrics_and_plots.ipynb to script\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Queued experiment 'factor_4' for future execution.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[NbConvertApp] Writing 3574 bytes to 04_metrics_and_plots.py\n",
      "[NbConvertApp] Converting notebook 04_metrics_and_plots.ipynb to script\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Queued experiment 'factor_5' for future execution.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[NbConvertApp] Writing 3574 bytes to 04_metrics_and_plots.py\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Following logs for all queued experiments. Use Ctrl+C to stop following logs (experiment execution will continue).\n",
      "\n",
      "Running stage 'MetricAndPlot':\n",
      "> zntrack run src.MetricAndPlot.MetricAndPlot --name MetricAndPlot\n",
      "Updating lock file 'dvc.lock'\n",
      "\n",
      "To track the changes with git, run:\n",
      "\n",
      "\tgit add dvc.yaml dvc.lock params.yaml\n",
      "\n",
      "To enable auto staging, run:\n",
      "\n",
      "\tdvc config core.autostage true\n",
      "Running stage 'MetricAndPlot':\n",
      "> zntrack run src.MetricAndPlot.MetricAndPlot --name MetricAndPlot\n",
      "Updating lock file 'dvc.lock'\n",
      "\n",
      "To track the changes with git, run:\n",
      "\n",
      "\tgit add dvc.lock params.yaml dvc.yaml\n",
      "\n",
      "To enable auto staging, run:\n",
      "\n",
      "\tdvc config core.autostage true\n",
      "Running stage 'MetricAndPlot':\n",
      "> zntrack run src.MetricAndPlot.MetricAndPlot --name MetricAndPlot\n",
      "Updating lock file 'dvc.lock'\n",
      "\n",
      "To track the changes with git, run:\n",
      "\n",
      "\tgit add dvc.yaml dvc.lock params.yaml\n",
      "\n",
      "To enable auto staging, run:\n",
      "\n",
      "\tdvc config core.autostage true\n",
      "Running stage 'MetricAndPlot':\n",
      "> zntrack run src.MetricAndPlot.MetricAndPlot --name MetricAndPlot\n",
      "Updating lock file 'dvc.lock'\n",
      "\n",
      "To track the changes with git, run:\n",
      "\n",
      "\tgit add dvc.lock params.yaml dvc.yaml\n",
      "\n",
      "To enable auto staging, run:\n",
      "\n",
      "\tdvc config core.autostage true\n",
      "\n",
      "Ran experiment(s): \n",
      "To apply the results of an experiment to your workspace run:\n",
      "\n",
      "\tdvc exp apply <exp>\n",
      "\n",
      "To promote an experiment to a Git branch run:\n",
      "\n",
      "\tdvc exp branch <exp> <branch>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with project.create_experiment(name=\"factor_2\"):\n",
    "    node.pre_factor = 2\n",
    "with project.create_experiment(name=\"factor_3\"):\n",
    "    node.pre_factor = 3\n",
    "with project.create_experiment(name=\"factor_4\"):\n",
    "    node.pre_factor = 4\n",
    "with project.create_experiment(name=\"factor_5\"):\n",
    "    node.pre_factor = 5\n",
    "\n",
    "project.run_exp()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8161ba6a-58b7-424f-bc4f-1f3933395840",
   "metadata": {},
   "source": [
    "Now that all experiments are done, we can look at the metrics directly with `dvc exp show` or `dvc metrics show/diff`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c0504b4d-6028-451c-bee1-d280c64fbc31",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rev</th>\n",
       "      <th>typ</th>\n",
       "      <th>Created</th>\n",
       "      <th>parent</th>\n",
       "      <th>metric_1</th>\n",
       "      <th>metric_2</th>\n",
       "      <th>MetricAndPlot.pre_factor</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Experiment</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>NaN</th>\n",
       "      <td>workspace</td>\n",
       "      <td>baseline</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>17.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>main</th>\n",
       "      <td>57992f7</td>\n",
       "      <td>baseline</td>\n",
       "      <td>2023-03-08T14:27:34</td>\n",
       "      <td>NaN</td>\n",
       "      <td>17.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor_5</th>\n",
       "      <td>8491765</td>\n",
       "      <td>branch_commit</td>\n",
       "      <td>2023-03-08T14:27:51</td>\n",
       "      <td>NaN</td>\n",
       "      <td>85.0</td>\n",
       "      <td>210.0</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor_4</th>\n",
       "      <td>646b1ad</td>\n",
       "      <td>branch_commit</td>\n",
       "      <td>2023-03-08T14:27:49</td>\n",
       "      <td>NaN</td>\n",
       "      <td>68.0</td>\n",
       "      <td>168.0</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor_3</th>\n",
       "      <td>efaea50</td>\n",
       "      <td>branch_commit</td>\n",
       "      <td>2023-03-08T14:27:46</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor_2</th>\n",
       "      <td>8790933</td>\n",
       "      <td>branch_base</td>\n",
       "      <td>2023-03-08T14:27:44</td>\n",
       "      <td>NaN</td>\n",
       "      <td>34.0</td>\n",
       "      <td>84.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  rev            typ              Created  parent  metric_1  \\\n",
       "Experiment                                                                    \n",
       "NaN         workspace       baseline                  NaN     NaN      17.0   \n",
       "main          57992f7       baseline  2023-03-08T14:27:34     NaN      17.0   \n",
       "factor_5      8491765  branch_commit  2023-03-08T14:27:51     NaN      85.0   \n",
       "factor_4      646b1ad  branch_commit  2023-03-08T14:27:49     NaN      68.0   \n",
       "factor_3      efaea50  branch_commit  2023-03-08T14:27:46     NaN      51.0   \n",
       "factor_2      8790933    branch_base  2023-03-08T14:27:44     NaN      34.0   \n",
       "\n",
       "            metric_2  MetricAndPlot.pre_factor  \n",
       "Experiment                                      \n",
       "NaN             42.0                       5.0  \n",
       "main            42.0                       1.0  \n",
       "factor_5       210.0                       5.0  \n",
       "factor_4       168.0                       4.0  \n",
       "factor_3       126.0                       3.0  \n",
       "factor_2        84.0                       2.0  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "!dvc exp show --csv > exp_show.csv\n",
    "\n",
    "pd.read_csv(\"exp_show.csv\", index_col=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5231be5f-9c74-4b3b-892d-65194a60fb5f",
   "metadata": {},
   "source": [
    "We can also use `dvc plots show/diff` to evaluate the plot data that we produced."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "606c0d93-a515-4b43-ab49-16aa5b40f0ba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "file:///tmp/tmpq5kwayap/dvc_plots/index.html\n"
     ]
    }
   ],
   "source": [
    "!dvc plots diff HEAD factor_2 factor_3 factor_4 factor_5"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d443f20-10ad-4f87-bd7c-bb3ba77e5f06",
   "metadata": {},
   "source": [
    "## The ZnTrack way\n",
    "\n",
    "ZnTrack provides and easier way to handle metrics. Similar to `zn.outs()` which does not require defining a path to outs file, one can use `zn.metrics`.\n",
    "The same is possible for plots via `zn.plots()`. To define additional [options](https://dvc.org/doc/command-reference/plots/modify) you can pass the keyword to `zn.plots()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "4db0df8b-2d63-4a59-a144-335e19c16d26",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "DeprecationWarning for write_graph: Building a graph is now done using 'with zntrack.Project() as project: ...' (Deprecated since 0.6.0)\n",
      "Running DVC command: 'stage add --name ZnTrackMetric --force ...'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Adding stage 'ZnTrackMetric' in 'dvc.yaml'\n",
      "\n",
      "To track the changes with git, run:\n",
      "\n",
      "\tgit add dvc.yaml nodes/ZnTrackMetric/.gitignore\n",
      "\n",
      "To enable auto staging, run:\n",
      "\n",
      "\tdvc config core.autostage true\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[NbConvertApp] Converting notebook 04_metrics_and_plots.ipynb to script\n",
      "[NbConvertApp] Writing 3574 bytes to 04_metrics_and_plots.py\n",
      "Running DVC command: 'repro ZnTrackMetric'\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running stage 'ZnTrackMetric':\n",
      "> zntrack run src.ZnTrackMetric.ZnTrackMetric --name ZnTrackMetric\n",
      "Updating lock file 'dvc.lock'\n",
      "\n",
      "To track the changes with git, run:\n",
      "\n",
      "\tgit add dvc.lock\n",
      "\n",
      "To enable auto staging, run:\n",
      "\n",
      "\tdvc config core.autostage true\n",
      "Use `dvc push` to send your updates to remote storage.\n"
     ]
    }
   ],
   "source": [
    "class ZnTrackMetric(zntrack.Node):\n",
    "    my_metric = zntrack.zn.metrics()\n",
    "    my_plot = zntrack.zn.plots()\n",
    "\n",
    "    def run(self):\n",
    "        self.my_metric = {\"alpha\": 1.0, \"beta\": 0.00473}\n",
    "        self.my_plot = pd.DataFrame({\"val\": np.sin(np.linspace(0, 3.14, 100))})\n",
    "        self.my_plot.index.name = (  # For DVC it is required that the index has a column name\n",
    "            \"index\"\n",
    "        )\n",
    "\n",
    "\n",
    "with zntrack.Project() as project:\n",
    "    node = ZnTrackMetric()\n",
    "\n",
    "project.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "f33d844c-c4bb-4726-85f7-49509056b7ef",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rev</th>\n",
       "      <th>typ</th>\n",
       "      <th>Created</th>\n",
       "      <th>parent</th>\n",
       "      <th>metric_1</th>\n",
       "      <th>metric_2</th>\n",
       "      <th>alpha</th>\n",
       "      <th>beta</th>\n",
       "      <th>MetricAndPlot.pre_factor</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Experiment</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>NaN</th>\n",
       "      <td>workspace</td>\n",
       "      <td>baseline</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>17.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.00473</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>main</th>\n",
       "      <td>57992f7</td>\n",
       "      <td>baseline</td>\n",
       "      <td>2023-03-08T14:27:34</td>\n",
       "      <td>NaN</td>\n",
       "      <td>17.0</td>\n",
       "      <td>42.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor_5</th>\n",
       "      <td>8491765</td>\n",
       "      <td>branch_commit</td>\n",
       "      <td>2023-03-08T14:27:51</td>\n",
       "      <td>NaN</td>\n",
       "      <td>85.0</td>\n",
       "      <td>210.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor_4</th>\n",
       "      <td>646b1ad</td>\n",
       "      <td>branch_commit</td>\n",
       "      <td>2023-03-08T14:27:49</td>\n",
       "      <td>NaN</td>\n",
       "      <td>68.0</td>\n",
       "      <td>168.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor_3</th>\n",
       "      <td>efaea50</td>\n",
       "      <td>branch_commit</td>\n",
       "      <td>2023-03-08T14:27:46</td>\n",
       "      <td>NaN</td>\n",
       "      <td>51.0</td>\n",
       "      <td>126.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>factor_2</th>\n",
       "      <td>8790933</td>\n",
       "      <td>branch_base</td>\n",
       "      <td>2023-03-08T14:27:44</td>\n",
       "      <td>NaN</td>\n",
       "      <td>34.0</td>\n",
       "      <td>84.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  rev            typ              Created  parent  metric_1  \\\n",
       "Experiment                                                                    \n",
       "NaN         workspace       baseline                  NaN     NaN      17.0   \n",
       "main          57992f7       baseline  2023-03-08T14:27:34     NaN      17.0   \n",
       "factor_5      8491765  branch_commit  2023-03-08T14:27:51     NaN      85.0   \n",
       "factor_4      646b1ad  branch_commit  2023-03-08T14:27:49     NaN      68.0   \n",
       "factor_3      efaea50  branch_commit  2023-03-08T14:27:46     NaN      51.0   \n",
       "factor_2      8790933    branch_base  2023-03-08T14:27:44     NaN      34.0   \n",
       "\n",
       "            metric_2  alpha     beta  MetricAndPlot.pre_factor  \n",
       "Experiment                                                      \n",
       "NaN             42.0    1.0  0.00473                       5.0  \n",
       "main            42.0    NaN      NaN                       1.0  \n",
       "factor_5       210.0    NaN      NaN                       5.0  \n",
       "factor_4       168.0    NaN      NaN                       4.0  \n",
       "factor_3       126.0    NaN      NaN                       3.0  \n",
       "factor_2        84.0    NaN      NaN                       2.0  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "!dvc exp show --csv > exp_show.csv\n",
    "\n",
    "pd.read_csv(\"exp_show.csv\", index_col=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4b9d3164-bc2f-49cf-a253-22c8f3efd7c0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "file:///tmp/tmpq5kwayap/dvc_plots/index.html\n"
     ]
    }
   ],
   "source": [
    "!dvc plots show"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "5b76227c",
   "metadata": {
    "collapsed": false,
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "outputs": [],
   "source": [
    "temp_dir.cleanup()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}