docs/build/html/tutorial_notebooks/deepof_supervised_tutorial.html
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.18.1: http://docutils.sourceforge.net/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Supervised annotation in DeepOF - search for pre-established patterns — deepof 0.6.2 documentation</title>
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="../_static/jupyter-sphinx.css" type="text/css" />
<link rel="stylesheet" href="../_static/thebelab.css" type="text/css" />
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
<link rel="shortcut icon" href="../_static/deepof.ico"/>
<!--[if lt IE 9]>
<script src="../_static/js/html5shiv.min.js"></script>
<![endif]-->
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/_sphinx_javascript_frameworks_compat.js"></script>
<script src="../_static/doctools.js"></script>
<script src="../_static/sphinx_highlight.js"></script>
<script src="../_static/thebelab-helper.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@jupyter-widgets/html-manager@^1.0.1/dist/embed-amd.js"></script>
<script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
<script>window.MathJax = {"tex": {"inlineMath": [["$", "$"], ["\\(", "\\)"]], "processEscapes": true}, "options": {"ignoreHtmlClass": "tex2jax_ignore|mathjax_ignore|document", "processHtmlClass": "tex2jax_process|mathjax_process|math|output_area"}}</script>
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
<script src="../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../index.html">
<img src="../_static/deepof_sidebar.ico" class="logo" alt="Logo"/>
</a>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<!-- Local TOC -->
<div class="local-toc"><ul>
<li><a class="reference internal" href="#">Supervised annotation in DeepOF - search for pre-established patterns</a><ul>
<li><a class="reference internal" href="#What-we’ll-cover:">What we’ll cover:</a><ul>
<li><a class="reference internal" href="#Loading-a-previously-initiated-project">Loading a previously initiated project</a></li>
<li><a class="reference internal" href="#Running-the-supervised-annotation-pipeline">Running the supervised annotation pipeline</a></li>
<li><a class="reference internal" href="#Generating-Gantt-charts-with-all-traits">Generating Gantt charts with all traits</a></li>
<li><a class="reference internal" href="#Exploring-trait-enrichment-across-conditions.">Exploring trait enrichment across conditions.</a></li>
<li><a class="reference internal" href="#PCA-embedding-of-supervised-traits">PCA embedding of supervised traits</a></li>
<li><a class="reference internal" href="#What’s-next">What’s next</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../index.html">deepof</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="../index.html" class="icon icon-home" aria-label="Home"></a></li>
<li class="breadcrumb-item active">Supervised annotation in DeepOF - search for pre-established patterns</li>
<li class="wy-breadcrumbs-aside">
<a href="../_sources/tutorial_notebooks/deepof_supervised_tutorial.ipynb.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<style>
/* CSS for nbsphinx extension */
/* remove conflicting styling from Sphinx themes */
div.nbinput.container div.prompt *,
div.nboutput.container div.prompt *,
div.nbinput.container div.input_area pre,
div.nboutput.container div.output_area pre,
div.nbinput.container div.input_area .highlight,
div.nboutput.container div.output_area .highlight {
border: none;
padding: 0;
margin: 0;
box-shadow: none;
}
div.nbinput.container > div[class*=highlight],
div.nboutput.container > div[class*=highlight] {
margin: 0;
}
div.nbinput.container div.prompt *,
div.nboutput.container div.prompt * {
background: none;
}
div.nboutput.container div.output_area .highlight,
div.nboutput.container div.output_area pre {
background: unset;
}
div.nboutput.container div.output_area div.highlight {
color: unset; /* override Pygments text color */
}
/* avoid gaps between output lines */
div.nboutput.container div[class*=highlight] pre {
line-height: normal;
}
/* input/output containers */
div.nbinput.container,
div.nboutput.container {
display: -webkit-flex;
display: flex;
align-items: flex-start;
margin: 0;
width: 100%;
}
@media (max-width: 540px) {
div.nbinput.container,
div.nboutput.container {
flex-direction: column;
}
}
/* input container */
div.nbinput.container {
padding-top: 5px;
}
/* last container */
div.nblast.container {
padding-bottom: 5px;
}
/* input prompt */
div.nbinput.container div.prompt pre {
color: #307FC1;
}
/* output prompt */
div.nboutput.container div.prompt pre {
color: #BF5B3D;
}
/* all prompts */
div.nbinput.container div.prompt,
div.nboutput.container div.prompt {
width: 4.5ex;
padding-top: 5px;
position: relative;
user-select: none;
}
div.nbinput.container div.prompt > div,
div.nboutput.container div.prompt > div {
position: absolute;
right: 0;
margin-right: 0.3ex;
}
@media (max-width: 540px) {
div.nbinput.container div.prompt,
div.nboutput.container div.prompt {
width: unset;
text-align: left;
padding: 0.4em;
}
div.nboutput.container div.prompt.empty {
padding: 0;
}
div.nbinput.container div.prompt > div,
div.nboutput.container div.prompt > div {
position: unset;
}
}
/* disable scrollbars and line breaks on prompts */
div.nbinput.container div.prompt pre,
div.nboutput.container div.prompt pre {
overflow: hidden;
white-space: pre;
}
/* input/output area */
div.nbinput.container div.input_area,
div.nboutput.container div.output_area {
-webkit-flex: 1;
flex: 1;
overflow: auto;
}
@media (max-width: 540px) {
div.nbinput.container div.input_area,
div.nboutput.container div.output_area {
width: 100%;
}
}
/* input area */
div.nbinput.container div.input_area {
border: 1px solid #e0e0e0;
border-radius: 2px;
/*background: #f5f5f5;*/
}
/* override MathJax center alignment in output cells */
div.nboutput.container div[class*=MathJax] {
text-align: left !important;
}
/* override sphinx.ext.imgmath center alignment in output cells */
div.nboutput.container div.math p {
text-align: left;
}
/* standard error */
div.nboutput.container div.output_area.stderr {
background: #fdd;
}
/* ANSI colors */
.ansi-black-fg { color: #3E424D; }
.ansi-black-bg { background-color: #3E424D; }
.ansi-black-intense-fg { color: #282C36; }
.ansi-black-intense-bg { background-color: #282C36; }
.ansi-red-fg { color: #E75C58; }
.ansi-red-bg { background-color: #E75C58; }
.ansi-red-intense-fg { color: #B22B31; }
.ansi-red-intense-bg { background-color: #B22B31; }
.ansi-green-fg { color: #00A250; }
.ansi-green-bg { background-color: #00A250; }
.ansi-green-intense-fg { color: #007427; }
.ansi-green-intense-bg { background-color: #007427; }
.ansi-yellow-fg { color: #DDB62B; }
.ansi-yellow-bg { background-color: #DDB62B; }
.ansi-yellow-intense-fg { color: #B27D12; }
.ansi-yellow-intense-bg { background-color: #B27D12; }
.ansi-blue-fg { color: #208FFB; }
.ansi-blue-bg { background-color: #208FFB; }
.ansi-blue-intense-fg { color: #0065CA; }
.ansi-blue-intense-bg { background-color: #0065CA; }
.ansi-magenta-fg { color: #D160C4; }
.ansi-magenta-bg { background-color: #D160C4; }
.ansi-magenta-intense-fg { color: #A03196; }
.ansi-magenta-intense-bg { background-color: #A03196; }
.ansi-cyan-fg { color: #60C6C8; }
.ansi-cyan-bg { background-color: #60C6C8; }
.ansi-cyan-intense-fg { color: #258F8F; }
.ansi-cyan-intense-bg { background-color: #258F8F; }
.ansi-white-fg { color: #C5C1B4; }
.ansi-white-bg { background-color: #C5C1B4; }
.ansi-white-intense-fg { color: #A1A6B2; }
.ansi-white-intense-bg { background-color: #A1A6B2; }
.ansi-default-inverse-fg { color: #FFFFFF; }
.ansi-default-inverse-bg { background-color: #000000; }
.ansi-bold { font-weight: bold; }
.ansi-underline { text-decoration: underline; }
div.nbinput.container div.input_area div[class*=highlight] > pre,
div.nboutput.container div.output_area div[class*=highlight] > pre,
div.nboutput.container div.output_area div[class*=highlight].math,
div.nboutput.container div.output_area.rendered_html,
div.nboutput.container div.output_area > div.output_javascript,
div.nboutput.container div.output_area:not(.rendered_html) > img{
padding: 5px;
margin: 0;
}
/* fix copybtn overflow problem in chromium (needed for 'sphinx_copybutton') */
div.nbinput.container div.input_area > div[class^='highlight'],
div.nboutput.container div.output_area > div[class^='highlight']{
overflow-y: hidden;
}
/* hide copybtn icon on prompts (needed for 'sphinx_copybutton') */
.prompt .copybtn {
display: none;
}
/* Some additional styling taken form the Jupyter notebook CSS */
.jp-RenderedHTMLCommon table,
div.rendered_html table {
border: none;
border-collapse: collapse;
border-spacing: 0;
color: black;
font-size: 12px;
table-layout: fixed;
}
.jp-RenderedHTMLCommon thead,
div.rendered_html thead {
border-bottom: 1px solid black;
vertical-align: bottom;
}
.jp-RenderedHTMLCommon tr,
.jp-RenderedHTMLCommon th,
.jp-RenderedHTMLCommon td,
div.rendered_html tr,
div.rendered_html th,
div.rendered_html td {
text-align: right;
vertical-align: middle;
padding: 0.5em 0.5em;
line-height: normal;
white-space: normal;
max-width: none;
border: none;
}
.jp-RenderedHTMLCommon th,
div.rendered_html th {
font-weight: bold;
}
.jp-RenderedHTMLCommon tbody tr:nth-child(odd),
div.rendered_html tbody tr:nth-child(odd) {
background: #f5f5f5;
}
.jp-RenderedHTMLCommon tbody tr:hover,
div.rendered_html tbody tr:hover {
background: rgba(66, 165, 245, 0.2);
}
/* CSS overrides for sphinx_rtd_theme */
/* 24px margin */
.nbinput.nblast.container,
.nboutput.nblast.container {
margin-bottom: 19px; /* padding has already 5px */
}
/* ... except between code cells! */
.nblast.container + .nbinput.container {
margin-top: -19px;
}
.admonition > p:before {
margin-right: 4px; /* make room for the exclamation icon */
}
/* Fix math alignment, see https://github.com/rtfd/sphinx_rtd_theme/pull/686 */
.math {
text-align: unset;
}
</style>
<section id="Supervised-annotation-in-DeepOF---search-for-pre-established-patterns">
<h1>Supervised annotation in DeepOF - search for pre-established patterns<a class="headerlink" href="#Supervised-annotation-in-DeepOF---search-for-pre-established-patterns" title="Permalink to this heading"></a></h1>
<p><a class="reference external" href="https://colab.research.google.com/github/lucasmiranda42/deepof/blob/master/docs/source/tutorial_notebooks/deepof_supervised_tutorial.ipynb"><img alt="Open In Colab" src="https://colab.research.google.com/assets/colab-badge.svg" /></a></p>
<section id="What-we’ll-cover:">
<h2>What we’ll cover:<a class="headerlink" href="#What-we’ll-cover:" title="Permalink to this heading"></a></h2>
<ul class="simple">
<li><p>Run the supervised annotation pipeline.</p></li>
<li><p>Generate Gantt plots with all traits.</p></li>
<li><p>Explore trait enrichment across conditions.</p></li>
<li><p>Visualize global embeddings using the retrieved traits.</p></li>
</ul>
<div class="nbinput nblast docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[1]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># # If using Google colab, uncomment and run this cell and the one below to set up the environment</span>
<span class="c1"># # Note: becasuse of how colab handles the installation of local packages, this cell will kill your runtime.</span>
<span class="c1"># # This is not an error! Just continue with the cells below.</span>
<span class="c1"># import os</span>
<span class="c1"># !git clone -q https://github.com/mlfpm/deepof.git</span>
<span class="c1"># !pip install -q -e deepof --progress-bar off</span>
<span class="c1"># os.chdir("deepof")</span>
<span class="c1"># !curl --output tutorial_files.zip https://datashare.mpcdf.mpg.de/s/knF7t78isQuIAr0/download</span>
<span class="c1"># !unzip tutorial_files.zip</span>
<span class="c1"># os.kill(os.getpid(), 9)</span>
</pre></div>
</div>
</div>
<div class="nbinput nblast docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[2]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># os.chdir("deepof")</span>
<span class="c1"># import os, warnings</span>
<span class="c1"># warnings.filterwarnings('ignore')</span>
</pre></div>
</div>
</div>
<p>Let’s start by importing some packages. We’ll use python’s os library to handle paths, pickle to load saved objects, and the data entry API within DeepOF, located in deepof.data</p>
<div class="nbinput nblast docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[3]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="kn">import</span> <span class="nn">deepof.data</span>
</pre></div>
</div>
</div>
<p>We’ll also need some plotting gear:</p>
<div class="nbinput nblast docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[4]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">deepof.visuals</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="kn">import</span> <span class="nn">seaborn</span> <span class="k">as</span> <span class="nn">sns</span>
</pre></div>
</div>
</div>
<section id="Loading-a-previously-initiated-project">
<h3>Loading a previously initiated project<a class="headerlink" href="#Loading-a-previously-initiated-project" title="Permalink to this heading"></a></h3>
<p>As last time, we’ll load the already created tutorial dataset for further processing:</p>
<div class="nbinput nblast docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[5]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="c1"># Load a previously saved project</span>
<span class="n">my_deepof_project</span> <span class="o">=</span> <span class="n">deepof</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">load_project</span><span class="p">(</span><span class="s2">"./tutorial_files/tutorial_project"</span><span class="p">)</span>
</pre></div>
</div>
</div>
<p><strong>NOTE</strong> to better show how DeepOF deals with statistics, all results shown in the documentation version of this tutorial were obtained using the full SI dataset, containing a total of 53 animals. If you’d like to gain access to this dataset, check out the code availability statement of the main DeepOF paper.</p>
</section>
<section id="Running-the-supervised-annotation-pipeline">
<h3>Running the supervised annotation pipeline<a class="headerlink" href="#Running-the-supervised-annotation-pipeline" title="Permalink to this heading"></a></h3>
<p>The supervised pipeline within DeepOF aims at providing simple but generalizable annotators for a variety of single and dyadic traits (as depicted in the figure below). Each of these traits is tagged independently, which means that more than one label can be applied per time point!</p>
<p><img alt="supervised" src="../_images/deepof_supervised.png" /></p>
<p>Running the pipeline is as simple as executing the <code class="docutils literal notranslate"><span class="pre">.supervised_annotation()</span></code> method in your Coordinates object. A progress bar will indicate how many videos have been processed already. Let’s run it and see what happens:</p>
<div class="nbinput docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[6]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">supervised_annotation</span> <span class="o">=</span> <span class="n">my_deepof_project</span><span class="o">.</span><span class="n">supervised_annotation</span><span class="p">()</span>
</pre></div>
</div>
</div>
<div class="nboutput nblast docutils container">
<div class="prompt empty docutils container">
</div>
<div class="output_area stderr docutils container">
<div class="highlight"><pre>
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:16<00:00, 2.74s/it]
</pre></div></div>
</div>
<p>It worked! Upon inspection, we see that supervised annotations contains a dictionary-like object with experiment IDs as keys, and data frames with annotations as values:</p>
<div class="nbinput docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[7]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">supervised_annotation</span><span class="p">[</span><span class="s1">'20191204_Day2_SI_JB08_Test_54'</span><span class="p">]</span>
</pre></div>
</div>
</div>
<div class="nboutput nblast docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[7]:
</pre></div>
</div>
<div class="output_area rendered_html docutils container">
<div>
<style scoped>
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
</style>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>B_W_nose2nose</th>
<th>B_W_sidebyside</th>
<th>B_W_sidereside</th>
<th>B_W_nose2tail</th>
<th>W_B_nose2tail</th>
<th>B_W_nose2body</th>
<th>W_B_nose2body</th>
<th>B_W_following</th>
<th>W_B_following</th>
<th>B_climbing</th>
<th>...</th>
<th>B_huddle</th>
<th>B_lookaround</th>
<th>B_speed</th>
<th>W_climbing</th>
<th>W_sniffing</th>
<th>W_huddle</th>
<th>W_lookaround</th>
<th>W_speed</th>
<th>B_missing</th>
<th>W_missing</th>
</tr>
</thead>
<tbody>
<tr>
<th>00:00:00</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>1.0</td>
<td>0.0</td>
<td>0.000</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.000</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>00:00:00.039935995</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>1.0</td>
<td>0.0</td>
<td>0.000</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.000</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>00:00:00.079871991</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>1.0</td>
<td>0.0</td>
<td>0.000</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.000</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>00:00:00.119807987</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.000</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.000</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>00:00:00.159743982</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>1.0</td>
<td>2.236</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>10.636</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>...</th>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
</tr>
<tr>
<th>00:09:58.800320021</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>6.593</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>1.0</td>
<td>0.086</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>00:09:58.840256017</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>4.934</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>1.0</td>
<td>0.083</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>00:09:58.880192012</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>5.557</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>1.0</td>
<td>0.064</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>00:09:58.920128008</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>6.130</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>1.0</td>
<td>0.046</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>00:09:58.960064004</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>4.628</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>0.031</td>
<td>0</td>
<td>0</td>
</tr>
</tbody>
</table>
<p>14999 rows × 21 columns</p>
</div></div>
</div>
<p>All annotations are tracked for both animals in the data (the column names have a prefix indicating to which animal they belong -B or W for black and white, respectively-). Labels with both prefixes (such as ‘B_W_nose2nose’) correspond to dyadic interactions. If you have more than two animals in your data, all pairwise combinations will appear as results.</p>
<p>Moreover, most cells in the data frame depicted above show 0 or 1 values, indicating the presence or absence of the given trait. The only exception is speed, which is expressed in millimeters per frame.</p>
<p><strong>NOTE</strong>: You’ll also find a column per animal (depicted with <code class="docutils literal notranslate"><span class="pre">ID_missing</span></code>) indicating whether it’s visible or not. This is useful to track when certain animals are completely occluded (for example, when they are in their nests).</p>
<p>Now that we know what we’re looking at, let’s explore a few simple analyses we can run within DeepOF:</p>
</section>
<section id="Generating-Gantt-charts-with-all-traits">
<h3>Generating Gantt charts with all traits<a class="headerlink" href="#Generating-Gantt-charts-with-all-traits" title="Permalink to this heading"></a></h3>
<p>The last thing we’ll do for now is to visualize all tagged patterns for a single video, using Gantt charts. To this end, let’s use deepof.visuals.plot_gantt()</p>
<div class="nbinput docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[8]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">8</span><span class="p">))</span>
<span class="n">deepof</span><span class="o">.</span><span class="n">visuals</span><span class="o">.</span><span class="n">plot_gantt</span><span class="p">(</span>
<span class="n">my_deepof_project</span><span class="p">,</span>
<span class="s1">'20191204_Day2_SI_JB08_Test_54'</span><span class="p">,</span>
<span class="n">supervised_annotations</span><span class="o">=</span><span class="n">supervised_annotation</span><span class="p">,</span>
<span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
</div>
</div>
<div class="nboutput nblast docutils container">
<div class="prompt empty docutils container">
</div>
<div class="output_area docutils container">
<img alt="../_images/tutorial_notebooks_deepof_supervised_tutorial_24_0.png" src="../_images/tutorial_notebooks_deepof_supervised_tutorial_24_0.png" />
</div>
</div>
<p>We see in these examples neither of the animals are ever occluded.</p>
<p>Thats good for a quick visualization, but let’s indeed compare how these patterns compare across experimental conditions!</p>
</section>
<section id="Exploring-trait-enrichment-across-conditions.">
<h3>Exploring trait enrichment across conditions.<a class="headerlink" href="#Exploring-trait-enrichment-across-conditions." title="Permalink to this heading"></a></h3>
<p>The simplest (but quite powerful) thing to do is to test for enrichment across traits. We can compute all statistics and plot the results using the <code class="docutils literal notranslate"><span class="pre">deepof.visuals.plot_enrichment()</span></code> function. The code below creates a figure with two plots with different scales (one for speed and one for everything else, given the differences in y-axis values).</p>
<div class="nbinput docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[8]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">fig</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">14</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span><span class="o">.</span><span class="n">subplot_mosaic</span><span class="p">(</span>
<span class="n">mosaic</span><span class="o">=</span><span class="s2">"""</span>
<span class="s2"> AAAAB</span>
<span class="s2"> AAAAB</span>
<span class="s2"> """</span><span class="p">,</span>
<span class="p">)</span>
<span class="n">deepof</span><span class="o">.</span><span class="n">visuals</span><span class="o">.</span><span class="n">plot_enrichment</span><span class="p">(</span>
<span class="n">my_deepof_project</span><span class="p">,</span>
<span class="n">supervised_annotations</span><span class="o">=</span><span class="n">supervised_annotation</span><span class="p">,</span>
<span class="n">add_stats</span><span class="o">=</span><span class="s2">"Mann-Whitney"</span><span class="p">,</span>
<span class="n">plot_proportions</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
<span class="n">ax</span> <span class="o">=</span> <span class="n">fig</span><span class="p">[</span><span class="s2">"A"</span><span class="p">],</span>
<span class="p">)</span>
<span class="n">deepof</span><span class="o">.</span><span class="n">visuals</span><span class="o">.</span><span class="n">plot_enrichment</span><span class="p">(</span>
<span class="n">my_deepof_project</span><span class="p">,</span>
<span class="n">supervised_annotations</span><span class="o">=</span><span class="n">supervised_annotation</span><span class="p">,</span>
<span class="n">add_stats</span><span class="o">=</span><span class="s2">"Mann-Whitney"</span><span class="p">,</span>
<span class="n">plot_proportions</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
<span class="n">ax</span> <span class="o">=</span> <span class="n">fig</span><span class="p">[</span><span class="s2">"B"</span><span class="p">],</span>
<span class="p">)</span>
<span class="k">for</span> <span class="n">ax</span> <span class="ow">in</span> <span class="n">fig</span><span class="p">:</span>
<span class="n">fig</span><span class="p">[</span><span class="n">ax</span><span class="p">]</span><span class="o">.</span><span class="n">set_xticklabels</span><span class="p">(</span><span class="n">fig</span><span class="p">[</span><span class="n">ax</span><span class="p">]</span><span class="o">.</span><span class="n">get_xticklabels</span><span class="p">(),</span> <span class="n">rotation</span><span class="o">=</span><span class="mi">45</span><span class="p">,</span> <span class="n">ha</span><span class="o">=</span><span class="s1">'right'</span><span class="p">)</span>
<span class="n">fig</span><span class="p">[</span><span class="n">ax</span><span class="p">]</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s2">""</span><span class="p">)</span>
<span class="n">fig</span><span class="p">[</span><span class="n">ax</span><span class="p">]</span><span class="o">.</span><span class="n">set_xlabel</span><span class="p">(</span><span class="s2">""</span><span class="p">)</span>
<span class="n">fig</span><span class="p">[</span><span class="s2">"A"</span><span class="p">]</span><span class="o">.</span><span class="n">get_legend</span><span class="p">()</span><span class="o">.</span><span class="n">remove</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">tight_layout</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
</div>
</div>
<div class="nboutput nblast docutils container">
<div class="prompt empty docutils container">
</div>
<div class="output_area docutils container">
<img alt="../_images/tutorial_notebooks_deepof_supervised_tutorial_29_0.png" src="../_images/tutorial_notebooks_deepof_supervised_tutorial_29_0.png" />
</div>
</div>
<p>We see DeepOF reports significant differences for many of these traits! If you’d like to print a detailed statistical summary, you can set the ‘verbose’ parameter to True in <code class="docutils literal notranslate"><span class="pre">deepof.visuals.plot_enrichment()</span></code>.</p>
<p>In the figures, you see all tagged traits in the X-axis, and the frames per video in the y-axis. Bars represent averages, and error bars standard deviations. By default, DeepOF runs a Mann-Whitney U non-parametric test on the means per test, and controls for multiple testing using Benjamini-Hochberg’s method. For details on how to change both the tests to run and the multiple testing parameters, feel free to check the full API reference or the function docstring.</p>
<p>Most functions within DeepOF allow the user to restrict the data to certain periods of time. This can be useful to quantify habituation, for example (as we’ll see), or to explore how the retrieved patterns interact with timed environmental cues (such as tones, experimental actions, etc.).</p>
<p>To see how this works in the context of supervised annotation enrichment, let’s generate the same figure as above, but for the first two minutes of data only:</p>
<div class="nbinput docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[9]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">fig</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">14</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span><span class="o">.</span><span class="n">subplot_mosaic</span><span class="p">(</span>
<span class="n">mosaic</span><span class="o">=</span><span class="s2">"""</span>
<span class="s2"> AAAAB</span>
<span class="s2"> AAAAB</span>
<span class="s2"> """</span><span class="p">,</span>
<span class="p">)</span>
<span class="n">deepof</span><span class="o">.</span><span class="n">visuals</span><span class="o">.</span><span class="n">plot_enrichment</span><span class="p">(</span>
<span class="n">my_deepof_project</span><span class="p">,</span>
<span class="n">supervised_annotations</span><span class="o">=</span><span class="n">supervised_annotation</span><span class="p">,</span>
<span class="n">add_stats</span><span class="o">=</span><span class="s2">"Mann-Whitney"</span><span class="p">,</span>
<span class="n">plot_proportions</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
<span class="n">bin_index</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span>
<span class="n">bin_size</span><span class="o">=</span><span class="mi">120</span><span class="p">,</span>
<span class="n">ax</span> <span class="o">=</span> <span class="n">fig</span><span class="p">[</span><span class="s2">"A"</span><span class="p">],</span>
<span class="p">)</span>
<span class="n">deepof</span><span class="o">.</span><span class="n">visuals</span><span class="o">.</span><span class="n">plot_enrichment</span><span class="p">(</span>
<span class="n">my_deepof_project</span><span class="p">,</span>
<span class="n">supervised_annotations</span><span class="o">=</span><span class="n">supervised_annotation</span><span class="p">,</span>
<span class="n">add_stats</span><span class="o">=</span><span class="s2">"Mann-Whitney"</span><span class="p">,</span>
<span class="n">plot_proportions</span><span class="o">=</span><span class="kc">False</span><span class="p">,</span>
<span class="n">bin_index</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span>
<span class="n">bin_size</span><span class="o">=</span><span class="mi">120</span><span class="p">,</span>
<span class="n">ax</span> <span class="o">=</span> <span class="n">fig</span><span class="p">[</span><span class="s2">"B"</span><span class="p">],</span>
<span class="p">)</span>
<span class="k">for</span> <span class="n">ax</span> <span class="ow">in</span> <span class="n">fig</span><span class="p">:</span>
<span class="n">fig</span><span class="p">[</span><span class="n">ax</span><span class="p">]</span><span class="o">.</span><span class="n">set_xticklabels</span><span class="p">(</span><span class="n">fig</span><span class="p">[</span><span class="n">ax</span><span class="p">]</span><span class="o">.</span><span class="n">get_xticklabels</span><span class="p">(),</span> <span class="n">rotation</span><span class="o">=</span><span class="mi">45</span><span class="p">,</span> <span class="n">ha</span><span class="o">=</span><span class="s1">'right'</span><span class="p">)</span>
<span class="n">fig</span><span class="p">[</span><span class="n">ax</span><span class="p">]</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s2">""</span><span class="p">)</span>
<span class="n">fig</span><span class="p">[</span><span class="n">ax</span><span class="p">]</span><span class="o">.</span><span class="n">set_xlabel</span><span class="p">(</span><span class="s2">""</span><span class="p">)</span>
<span class="n">fig</span><span class="p">[</span><span class="s2">"A"</span><span class="p">]</span><span class="o">.</span><span class="n">get_legend</span><span class="p">()</span><span class="o">.</span><span class="n">remove</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">tight_layout</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
</div>
</div>
<div class="nboutput nblast docutils container">
<div class="prompt empty docutils container">
</div>
<div class="output_area docutils container">
<img alt="../_images/tutorial_notebooks_deepof_supervised_tutorial_32_0.png" src="../_images/tutorial_notebooks_deepof_supervised_tutorial_32_0.png" />
</div>
</div>
<p>Even though at first glance the figure may look similar, there are some important differences! For starters, the y-axis now shows lower values, since we’re only looking at two minutes of data instead of ten. Moreover, the effect sizes in some tagged patterns (such as huddling or speed) are much larger. Some patterns (such as nose2body) even show high significance where they were barely different across conditions before!</p>
<p>We seem to already be picking up the habituation of the stressed animals to their novel environment. For more details on this specific effect and the rationale behind how it’s quantified, feel free to check out the main DeepOF paper!</p>
<p>But, for now, let’s continue with another interesting tool: PCA projections.</p>
</section>
<section id="PCA-embedding-of-supervised-traits">
<h3>PCA embedding of supervised traits<a class="headerlink" href="#PCA-embedding-of-supervised-traits" title="Permalink to this heading"></a></h3>
<p>An often useful way of looking at multivariate data (such as the annotation vectors we just generated) is to project them in the lower-dimensional space. We’ll explore more sophisticated ways of doing that in the next tutorial, but for now we can just run these vectors through PCA. DeepOF allows us to do this with the <code class="docutils literal notranslate"><span class="pre">deepof.visuals.plot_embeddings()</span></code> function.</p>
<p>Moreover, we can also restrict the analysis over time. Let’s compare, again, the whole data to the first two minutes:</p>
<div class="nbinput docutils container">
<div class="prompt highlight-none notranslate"><div class="highlight"><pre><span></span>[14]:
</pre></div>
</div>
<div class="input_area highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="n">fig</span><span class="p">,</span> <span class="p">(</span><span class="n">ax1</span><span class="p">,</span> <span class="n">ax2</span><span class="p">)</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">6</span><span class="p">))</span>
<span class="n">deepof</span><span class="o">.</span><span class="n">visuals</span><span class="o">.</span><span class="n">plot_embeddings</span><span class="p">(</span>
<span class="n">my_deepof_project</span><span class="p">,</span>
<span class="n">supervised_annotations</span><span class="o">=</span><span class="n">supervised_annotation</span><span class="p">,</span>
<span class="n">ax</span><span class="o">=</span><span class="n">ax1</span><span class="p">,</span>
<span class="p">)</span>
<span class="n">deepof</span><span class="o">.</span><span class="n">visuals</span><span class="o">.</span><span class="n">plot_embeddings</span><span class="p">(</span>
<span class="n">my_deepof_project</span><span class="p">,</span>
<span class="n">supervised_annotations</span><span class="o">=</span><span class="n">supervised_annotation</span><span class="p">,</span>
<span class="n">bin_size</span><span class="o">=</span><span class="mi">120</span><span class="p">,</span>
<span class="n">bin_index</span><span class="o">=</span><span class="mi">0</span><span class="p">,</span>
<span class="n">ax</span><span class="o">=</span><span class="n">ax2</span><span class="p">,</span>
<span class="p">)</span>
<span class="n">ax1</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s2">"supervised embeddings of full videos"</span><span class="p">)</span>
<span class="n">ax2</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s2">"supervised embeddings of first two minutes"</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">tight_layout</span><span class="p">()</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>
</div>
</div>
<div class="nboutput nblast docutils container">
<div class="prompt empty docutils container">
</div>
<div class="output_area docutils container">
<img alt="../_images/tutorial_notebooks_deepof_supervised_tutorial_36_0.png" src="../_images/tutorial_notebooks_deepof_supervised_tutorial_36_0.png" />
</div>
</div>
<p>We can see how, indeed, the PCA on the average set of annotators seems to distinguish across conditions better when looking at the first two minutes only. We’ll continue to explore this idea in the next (and last) tutorial.</p>
</section>
<section id="What’s-next">
<h3>What’s next<a class="headerlink" href="#What’s-next" title="Permalink to this heading"></a></h3>
<p>That’s it for this second tutorial. <a class="reference external" href="https://deepof.readthedocs.io/en/latest/tutorial_notebooks/deepof_unsupervised_tutorial.html">Next</a> and last for now, we’ll explore the main forte of the package: we’ll learn to embed our data in a sequence-aware manner, to get behavioral patterns in an unsupervised way using deep clustering (where the “deep” in DeepOF comes from). See you there!</p>
</section>
</section>
</section>
</div>
</div>
<footer>
<hr/>
<div role="contentinfo">
<p>© Copyright 2024, Lucas Miranda.</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>