python/examples/basic/Logging_Different_Data.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
">### 🚩 *Create a free WhyLabs account to get more value out of whylogs!*<br> \n",
">*Did you know you can store, visualize, and monitor whylogs profiles with the [WhyLabs Observability Platform](https://whylabs.ai/whylogs-free-signup?utm_source=whylogs-Github&utm_medium=whylogs-example&utm_campaign=Logging_Different_Data)? Sign up for a [free WhyLabs account](https://whylabs.ai/whylogs-free-signup?utm_source=whylogs-Github&utm_medium=whylogs-example&utm_campaign=Logging_Different_Data) to leverage the power of whylogs and WhyLabs together!*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# (why)Logging"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[](https://colab.research.google.com/github/whylabs/whylogs/blob/mainline/python/examples/basic/Logging_Different_Data.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true,
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"WhyLogs enables logging different types of data that can then be used to monitor the data. We'll go through examples on different types of data to log and go more in depth on different options. Before we get going though, let's import whylogs."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[33mWARNING: You are using pip version 22.0.3; however, version 22.1 is available.\r\n",
"You should consider upgrading via the '/Users/melanie/Dev/whylogs-v1/python/.venv/bin/python -m pip install --upgrade pip' command.\u001b[0m\u001b[33m\r\n",
"\u001b[0m"
]
}
],
"source": [
"# Note: you may need to restart the kernel to use updated packages.\n",
"%pip install whylogs"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"## Table of Contents\n",
"#### [Log Pandas Dataframe](#log_dataframe) | [Log Dictionary](#log_dict) | [Display Logs](#display)"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"## Log Pandas DataFrame <a class=\"anchor\" id=\"log_dataframe\"></a>\n",
"\n",
"We will be generating log by importing data from a CSV into Pandas Dataframe, logging it with the whylogs python library."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Transaction ID</th>\n",
" <th>Customer ID</th>\n",
" <th>Product Subcategory Code</th>\n",
" <th>Product Category Code</th>\n",
" <th>Item Price</th>\n",
" <th>Total Tax</th>\n",
" <th>Total Amount</th>\n",
" <th>Store Type</th>\n",
" <th>Product Category</th>\n",
" <th>Product Subcategory</th>\n",
" <th>Date of Birth</th>\n",
" <th>Gender</th>\n",
" <th>City Code</th>\n",
" <th>Age at Transaction Date</th>\n",
" <th>Purchase Canceled</th>\n",
" <th>Transaction Day of Week</th>\n",
" <th>Transaction Week</th>\n",
" <th>Transaction Batch</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>T25601292314</td>\n",
" <td>C268458</td>\n",
" <td>12</td>\n",
" <td>6</td>\n",
" <td>114.9</td>\n",
" <td>24.1290</td>\n",
" <td>253.9290</td>\n",
" <td>TeleShop</td>\n",
" <td>Home and kitchen</td>\n",
" <td>Tools</td>\n",
" <td>1976-10-08</td>\n",
" <td>M</td>\n",
" <td>1.0</td>\n",
" <td>36.0</td>\n",
" <td>0.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>T1465175267</td>\n",
" <td>C271344</td>\n",
" <td>3</td>\n",
" <td>5</td>\n",
" <td>107.7</td>\n",
" <td>22.6170</td>\n",
" <td>238.0170</td>\n",
" <td>e-Shop</td>\n",
" <td>Books</td>\n",
" <td>Comics</td>\n",
" <td>1970-01-29</td>\n",
" <td>F</td>\n",
" <td>5.0</td>\n",
" <td>43.0</td>\n",
" <td>0.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>T4968790114</td>\n",
" <td>C272305</td>\n",
" <td>4</td>\n",
" <td>3</td>\n",
" <td>14.6</td>\n",
" <td>7.6650</td>\n",
" <td>80.6650</td>\n",
" <td>e-Shop</td>\n",
" <td>Electronics</td>\n",
" <td>Mobiles</td>\n",
" <td>1975-08-25</td>\n",
" <td>F</td>\n",
" <td>10.0</td>\n",
" <td>37.0</td>\n",
" <td>0.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>T50504166310</td>\n",
" <td>C275057</td>\n",
" <td>4</td>\n",
" <td>4</td>\n",
" <td>15.7</td>\n",
" <td>4.9455</td>\n",
" <td>52.0455</td>\n",
" <td>MBR</td>\n",
" <td>Bags</td>\n",
" <td>Women</td>\n",
" <td>1980-09-17</td>\n",
" <td>M</td>\n",
" <td>7.0</td>\n",
" <td>32.0</td>\n",
" <td>0.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>T10877729712</td>\n",
" <td>C270074</td>\n",
" <td>10</td>\n",
" <td>5</td>\n",
" <td>144.1</td>\n",
" <td>45.3915</td>\n",
" <td>477.6915</td>\n",
" <td>e-Shop</td>\n",
" <td>Books</td>\n",
" <td>Non-Fiction</td>\n",
" <td>1983-02-20</td>\n",
" <td>M</td>\n",
" <td>10.0</td>\n",
" <td>30.0</td>\n",
" <td>0.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>904</th>\n",
" <td>T89167826318</td>\n",
" <td>C274270</td>\n",
" <td>10</td>\n",
" <td>5</td>\n",
" <td>68.2</td>\n",
" <td>14.3220</td>\n",
" <td>150.7220</td>\n",
" <td>Flagship store</td>\n",
" <td>Books</td>\n",
" <td>Non-Fiction</td>\n",
" <td>1972-06-06</td>\n",
" <td>F</td>\n",
" <td>1.0</td>\n",
" <td>40.0</td>\n",
" <td>0.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>905</th>\n",
" <td>T87193008634</td>\n",
" <td>C271051</td>\n",
" <td>11</td>\n",
" <td>6</td>\n",
" <td>124.2</td>\n",
" <td>52.1640</td>\n",
" <td>548.9640</td>\n",
" <td>e-Shop</td>\n",
" <td>Home and kitchen</td>\n",
" <td>Bath</td>\n",
" <td>1976-02-13</td>\n",
" <td>F</td>\n",
" <td>3.0</td>\n",
" <td>36.0</td>\n",
" <td>0.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>906</th>\n",
" <td>T84036395834</td>\n",
" <td>C270763</td>\n",
" <td>11</td>\n",
" <td>5</td>\n",
" <td>77.9</td>\n",
" <td>16.3590</td>\n",
" <td>172.1590</td>\n",
" <td>TeleShop</td>\n",
" <td>Books</td>\n",
" <td>Children</td>\n",
" <td>1991-02-10</td>\n",
" <td>F</td>\n",
" <td>8.0</td>\n",
" <td>21.0</td>\n",
" <td>0.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>907</th>\n",
" <td>T72150045625</td>\n",
" <td>C270432</td>\n",
" <td>11</td>\n",
" <td>5</td>\n",
" <td>11.8</td>\n",
" <td>4.9560</td>\n",
" <td>52.1560</td>\n",
" <td>e-Shop</td>\n",
" <td>Books</td>\n",
" <td>Children</td>\n",
" <td>1982-09-17</td>\n",
" <td>M</td>\n",
" <td>7.0</td>\n",
" <td>30.0</td>\n",
" <td>1.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>908</th>\n",
" <td>T97942600110</td>\n",
" <td>C269559</td>\n",
" <td>4</td>\n",
" <td>3</td>\n",
" <td>67.9</td>\n",
" <td>35.6475</td>\n",
" <td>375.1475</td>\n",
" <td>e-Shop</td>\n",
" <td>Electronics</td>\n",
" <td>Mobiles</td>\n",
" <td>1972-06-24</td>\n",
" <td>M</td>\n",
" <td>1.0</td>\n",
" <td>40.0</td>\n",
" <td>0.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>909 rows × 18 columns</p>\n",
"</div>"
],
"text/plain": [
" Transaction ID Customer ID Product Subcategory Code \\\n",
"0 T25601292314 C268458 12 \n",
"1 T1465175267 C271344 3 \n",
"2 T4968790114 C272305 4 \n",
"3 T50504166310 C275057 4 \n",
"4 T10877729712 C270074 10 \n",
".. ... ... ... \n",
"904 T89167826318 C274270 10 \n",
"905 T87193008634 C271051 11 \n",
"906 T84036395834 C270763 11 \n",
"907 T72150045625 C270432 11 \n",
"908 T97942600110 C269559 4 \n",
"\n",
" Product Category Code Item Price Total Tax Total Amount \\\n",
"0 6 114.9 24.1290 253.9290 \n",
"1 5 107.7 22.6170 238.0170 \n",
"2 3 14.6 7.6650 80.6650 \n",
"3 4 15.7 4.9455 52.0455 \n",
"4 5 144.1 45.3915 477.6915 \n",
".. ... ... ... ... \n",
"904 5 68.2 14.3220 150.7220 \n",
"905 6 124.2 52.1640 548.9640 \n",
"906 5 77.9 16.3590 172.1590 \n",
"907 5 11.8 4.9560 52.1560 \n",
"908 3 67.9 35.6475 375.1475 \n",
"\n",
" Store Type Product Category Product Subcategory Date of Birth \\\n",
"0 TeleShop Home and kitchen Tools 1976-10-08 \n",
"1 e-Shop Books Comics 1970-01-29 \n",
"2 e-Shop Electronics Mobiles 1975-08-25 \n",
"3 MBR Bags Women 1980-09-17 \n",
"4 e-Shop Books Non-Fiction 1983-02-20 \n",
".. ... ... ... ... \n",
"904 Flagship store Books Non-Fiction 1972-06-06 \n",
"905 e-Shop Home and kitchen Bath 1976-02-13 \n",
"906 TeleShop Books Children 1991-02-10 \n",
"907 e-Shop Books Children 1982-09-17 \n",
"908 e-Shop Electronics Mobiles 1972-06-24 \n",
"\n",
" Gender City Code Age at Transaction Date Purchase Canceled \\\n",
"0 M 1.0 36.0 0.0 \n",
"1 F 5.0 43.0 0.0 \n",
"2 F 10.0 37.0 0.0 \n",
"3 M 7.0 32.0 0.0 \n",
"4 M 10.0 30.0 0.0 \n",
".. ... ... ... ... \n",
"904 F 1.0 40.0 0.0 \n",
"905 F 3.0 36.0 0.0 \n",
"906 F 8.0 21.0 0.0 \n",
"907 M 7.0 30.0 1.0 \n",
"908 M 1.0 40.0 0.0 \n",
"\n",
" Transaction Day of Week Transaction Week Transaction Batch \n",
"0 0 0 0 \n",
"1 0 0 0 \n",
"2 0 0 0 \n",
"3 0 0 0 \n",
"4 0 0 0 \n",
".. ... ... ... \n",
"904 0 0 0 \n",
"905 0 0 0 \n",
"906 0 0 0 \n",
"907 0 0 0 \n",
"908 0 0 0 \n",
"\n",
"[909 rows x 18 columns]"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import os.path\n",
"import pandas as pd\n",
"\n",
"# Read in a CSV, this one is from a public bucket on s3\n",
"retail_daily = pd.read_csv('https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/retail-daily-features.csv')\n",
"retail_daily"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>counts/n</th>\n",
" <th>counts/null</th>\n",
" <th>types/integral</th>\n",
" <th>types/fractional</th>\n",
" <th>types/boolean</th>\n",
" <th>types/string</th>\n",
" <th>types/object</th>\n",
" <th>cardinality/est</th>\n",
" <th>cardinality/upper_1</th>\n",
" <th>cardinality/lower_1</th>\n",
" <th>...</th>\n",
" <th>distribution/min</th>\n",
" <th>distribution/q_10</th>\n",
" <th>distribution/q_25</th>\n",
" <th>distribution/median</th>\n",
" <th>distribution/q_75</th>\n",
" <th>distribution/q_90</th>\n",
" <th>type</th>\n",
" <th>ints/max</th>\n",
" <th>ints/min</th>\n",
" <th>frequent_items/frequent_strings</th>\n",
" </tr>\n",
" <tr>\n",
" <th>column</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>Purchase Canceled</th>\n",
" <td>909</td>\n",
" <td>72</td>\n",
" <td>0</td>\n",
" <td>837</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>2.000000</td>\n",
" <td>2.000100</td>\n",
" <td>2.000000</td>\n",
" <td>...</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Age at Transaction Date</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>25.000001</td>\n",
" <td>25.001250</td>\n",
" <td>25.000000</td>\n",
" <td>...</td>\n",
" <td>19.000</td>\n",
" <td>21.0000</td>\n",
" <td>25.000</td>\n",
" <td>31.0000</td>\n",
" <td>37.0000</td>\n",
" <td>40.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Transaction Week</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1.000000</td>\n",
" <td>1.000050</td>\n",
" <td>1.000000</td>\n",
" <td>...</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>[FrequentItem(value='0.000000', est=909, upper...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Store Type</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>4.000000</td>\n",
" <td>4.000200</td>\n",
" <td>4.000000</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='e-Shop', est=375, upper=3...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Product Category</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>6.000000</td>\n",
" <td>6.000300</td>\n",
" <td>6.000000</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='Books', est=232, upper=23...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Gender</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>2.000000</td>\n",
" <td>2.000100</td>\n",
" <td>2.000000</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='M', est=455, upper=455, l...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Transaction ID</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>904.722898</td>\n",
" <td>916.565225</td>\n",
" <td>893.168643</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='T40336799311', est=3, upp...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Item Price</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>672.542875</td>\n",
" <td>681.346093</td>\n",
" <td>663.953801</td>\n",
" <td>...</td>\n",
" <td>7.100</td>\n",
" <td>18.2000</td>\n",
" <td>43.200</td>\n",
" <td>80.1000</td>\n",
" <td>116.2000</td>\n",
" <td>137.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Total Tax</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>800.975225</td>\n",
" <td>811.459552</td>\n",
" <td>790.745935</td>\n",
" <td>...</td>\n",
" <td>0.861</td>\n",
" <td>4.8825</td>\n",
" <td>10.017</td>\n",
" <td>20.5170</td>\n",
" <td>36.1725</td>\n",
" <td>54.2640</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Product Category Code</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>6.000000</td>\n",
" <td>6.000300</td>\n",
" <td>6.000000</td>\n",
" <td>...</td>\n",
" <td>1.000</td>\n",
" <td>1.0000</td>\n",
" <td>2.000</td>\n",
" <td>4.0000</td>\n",
" <td>5.0000</td>\n",
" <td>6.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>6.0</td>\n",
" <td>1.0</td>\n",
" <td>[FrequentItem(value='5.000000', est=232, upper...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Transaction Day of Week</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1.000000</td>\n",
" <td>1.000050</td>\n",
" <td>1.000000</td>\n",
" <td>...</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>[FrequentItem(value='0.000000', est=909, upper...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>City Code</th>\n",
" <td>909</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>908</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>10.000000</td>\n",
" <td>10.000500</td>\n",
" <td>10.000000</td>\n",
" <td>...</td>\n",
" <td>1.000</td>\n",
" <td>1.0000</td>\n",
" <td>3.000</td>\n",
" <td>5.0000</td>\n",
" <td>8.0000</td>\n",
" <td>10.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Transaction Batch</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1.000000</td>\n",
" <td>1.000050</td>\n",
" <td>1.000000</td>\n",
" <td>...</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>[FrequentItem(value='0.000000', est=909, upper...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Date of Birth</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>801.978113</td>\n",
" <td>812.475567</td>\n",
" <td>791.736016</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='1981-03-29', est=4, upper...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Customer ID</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>847.420398</td>\n",
" <td>858.512667</td>\n",
" <td>836.597955</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='C274278', est=4, upper=3,...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Total Amount</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>842.098548</td>\n",
" <td>853.121157</td>\n",
" <td>831.344071</td>\n",
" <td>...</td>\n",
" <td>-767.975</td>\n",
" <td>25.3045</td>\n",
" <td>82.433</td>\n",
" <td>188.4025</td>\n",
" <td>358.6830</td>\n",
" <td>555.2625</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Product Subcategory</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>18.000001</td>\n",
" <td>18.000899</td>\n",
" <td>18.000000</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='Women', est=133, upper=13...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Product Subcategory Code</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>12.000000</td>\n",
" <td>12.000599</td>\n",
" <td>12.000000</td>\n",
" <td>...</td>\n",
" <td>1.000</td>\n",
" <td>1.0000</td>\n",
" <td>3.000</td>\n",
" <td>5.0000</td>\n",
" <td>10.0000</td>\n",
" <td>11.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>12.0</td>\n",
" <td>1.0</td>\n",
" <td>[FrequentItem(value='4.000000', est=148, upper...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>18 rows × 24 columns</p>\n",
"</div>"
],
"text/plain": [
" counts/n counts/null types/integral \\\n",
"column \n",
"Purchase Canceled 909 72 0 \n",
"Age at Transaction Date 909 0 0 \n",
"Transaction Week 909 0 909 \n",
"Store Type 909 0 0 \n",
"Product Category 909 0 0 \n",
"Gender 909 0 0 \n",
"Transaction ID 909 0 0 \n",
"Item Price 909 0 0 \n",
"Total Tax 909 0 0 \n",
"Product Category Code 909 0 909 \n",
"Transaction Day of Week 909 0 909 \n",
"City Code 909 1 0 \n",
"Transaction Batch 909 0 909 \n",
"Date of Birth 909 0 0 \n",
"Customer ID 909 0 0 \n",
"Total Amount 909 0 0 \n",
"Product Subcategory 909 0 0 \n",
"Product Subcategory Code 909 0 909 \n",
"\n",
" types/fractional types/boolean types/string \\\n",
"column \n",
"Purchase Canceled 837 0 0 \n",
"Age at Transaction Date 909 0 0 \n",
"Transaction Week 0 0 0 \n",
"Store Type 0 0 909 \n",
"Product Category 0 0 909 \n",
"Gender 0 0 909 \n",
"Transaction ID 0 0 909 \n",
"Item Price 909 0 0 \n",
"Total Tax 909 0 0 \n",
"Product Category Code 0 0 0 \n",
"Transaction Day of Week 0 0 0 \n",
"City Code 908 0 0 \n",
"Transaction Batch 0 0 0 \n",
"Date of Birth 0 0 909 \n",
"Customer ID 0 0 909 \n",
"Total Amount 909 0 0 \n",
"Product Subcategory 0 0 909 \n",
"Product Subcategory Code 0 0 0 \n",
"\n",
" types/object cardinality/est cardinality/upper_1 \\\n",
"column \n",
"Purchase Canceled 0 2.000000 2.000100 \n",
"Age at Transaction Date 0 25.000001 25.001250 \n",
"Transaction Week 0 1.000000 1.000050 \n",
"Store Type 0 4.000000 4.000200 \n",
"Product Category 0 6.000000 6.000300 \n",
"Gender 0 2.000000 2.000100 \n",
"Transaction ID 0 904.722898 916.565225 \n",
"Item Price 0 672.542875 681.346093 \n",
"Total Tax 0 800.975225 811.459552 \n",
"Product Category Code 0 6.000000 6.000300 \n",
"Transaction Day of Week 0 1.000000 1.000050 \n",
"City Code 0 10.000000 10.000500 \n",
"Transaction Batch 0 1.000000 1.000050 \n",
"Date of Birth 0 801.978113 812.475567 \n",
"Customer ID 0 847.420398 858.512667 \n",
"Total Amount 0 842.098548 853.121157 \n",
"Product Subcategory 0 18.000001 18.000899 \n",
"Product Subcategory Code 0 12.000000 12.000599 \n",
"\n",
" cardinality/lower_1 ... distribution/min \\\n",
"column ... \n",
"Purchase Canceled 2.000000 ... 0.000 \n",
"Age at Transaction Date 25.000000 ... 19.000 \n",
"Transaction Week 1.000000 ... 0.000 \n",
"Store Type 4.000000 ... NaN \n",
"Product Category 6.000000 ... NaN \n",
"Gender 2.000000 ... NaN \n",
"Transaction ID 893.168643 ... NaN \n",
"Item Price 663.953801 ... 7.100 \n",
"Total Tax 790.745935 ... 0.861 \n",
"Product Category Code 6.000000 ... 1.000 \n",
"Transaction Day of Week 1.000000 ... 0.000 \n",
"City Code 10.000000 ... 1.000 \n",
"Transaction Batch 1.000000 ... 0.000 \n",
"Date of Birth 791.736016 ... NaN \n",
"Customer ID 836.597955 ... NaN \n",
"Total Amount 831.344071 ... -767.975 \n",
"Product Subcategory 18.000000 ... NaN \n",
"Product Subcategory Code 12.000000 ... 1.000 \n",
"\n",
" distribution/q_10 distribution/q_25 \\\n",
"column \n",
"Purchase Canceled 0.0000 0.000 \n",
"Age at Transaction Date 21.0000 25.000 \n",
"Transaction Week 0.0000 0.000 \n",
"Store Type NaN NaN \n",
"Product Category NaN NaN \n",
"Gender NaN NaN \n",
"Transaction ID NaN NaN \n",
"Item Price 18.2000 43.200 \n",
"Total Tax 4.8825 10.017 \n",
"Product Category Code 1.0000 2.000 \n",
"Transaction Day of Week 0.0000 0.000 \n",
"City Code 1.0000 3.000 \n",
"Transaction Batch 0.0000 0.000 \n",
"Date of Birth NaN NaN \n",
"Customer ID NaN NaN \n",
"Total Amount 25.3045 82.433 \n",
"Product Subcategory NaN NaN \n",
"Product Subcategory Code 1.0000 3.000 \n",
"\n",
" distribution/median distribution/q_75 \\\n",
"column \n",
"Purchase Canceled 0.0000 0.0000 \n",
"Age at Transaction Date 31.0000 37.0000 \n",
"Transaction Week 0.0000 0.0000 \n",
"Store Type NaN NaN \n",
"Product Category NaN NaN \n",
"Gender NaN NaN \n",
"Transaction ID NaN NaN \n",
"Item Price 80.1000 116.2000 \n",
"Total Tax 20.5170 36.1725 \n",
"Product Category Code 4.0000 5.0000 \n",
"Transaction Day of Week 0.0000 0.0000 \n",
"City Code 5.0000 8.0000 \n",
"Transaction Batch 0.0000 0.0000 \n",
"Date of Birth NaN NaN \n",
"Customer ID NaN NaN \n",
"Total Amount 188.4025 358.6830 \n",
"Product Subcategory NaN NaN \n",
"Product Subcategory Code 5.0000 10.0000 \n",
"\n",
" distribution/q_90 type ints/max \\\n",
"column \n",
"Purchase Canceled 0.0000 SummaryType.COLUMN NaN \n",
"Age at Transaction Date 40.0000 SummaryType.COLUMN NaN \n",
"Transaction Week 0.0000 SummaryType.COLUMN 0.0 \n",
"Store Type NaN SummaryType.COLUMN NaN \n",
"Product Category NaN SummaryType.COLUMN NaN \n",
"Gender NaN SummaryType.COLUMN NaN \n",
"Transaction ID NaN SummaryType.COLUMN NaN \n",
"Item Price 137.0000 SummaryType.COLUMN NaN \n",
"Total Tax 54.2640 SummaryType.COLUMN NaN \n",
"Product Category Code 6.0000 SummaryType.COLUMN 6.0 \n",
"Transaction Day of Week 0.0000 SummaryType.COLUMN 0.0 \n",
"City Code 10.0000 SummaryType.COLUMN NaN \n",
"Transaction Batch 0.0000 SummaryType.COLUMN 0.0 \n",
"Date of Birth NaN SummaryType.COLUMN NaN \n",
"Customer ID NaN SummaryType.COLUMN NaN \n",
"Total Amount 555.2625 SummaryType.COLUMN NaN \n",
"Product Subcategory NaN SummaryType.COLUMN NaN \n",
"Product Subcategory Code 11.0000 SummaryType.COLUMN 12.0 \n",
"\n",
" ints/min \\\n",
"column \n",
"Purchase Canceled NaN \n",
"Age at Transaction Date NaN \n",
"Transaction Week 0.0 \n",
"Store Type NaN \n",
"Product Category NaN \n",
"Gender NaN \n",
"Transaction ID NaN \n",
"Item Price NaN \n",
"Total Tax NaN \n",
"Product Category Code 1.0 \n",
"Transaction Day of Week 0.0 \n",
"City Code NaN \n",
"Transaction Batch 0.0 \n",
"Date of Birth NaN \n",
"Customer ID NaN \n",
"Total Amount NaN \n",
"Product Subcategory NaN \n",
"Product Subcategory Code 1.0 \n",
"\n",
" frequent_items/frequent_strings \n",
"column \n",
"Purchase Canceled NaN \n",
"Age at Transaction Date NaN \n",
"Transaction Week [FrequentItem(value='0.000000', est=909, upper... \n",
"Store Type [FrequentItem(value='e-Shop', est=375, upper=3... \n",
"Product Category [FrequentItem(value='Books', est=232, upper=23... \n",
"Gender [FrequentItem(value='M', est=455, upper=455, l... \n",
"Transaction ID [FrequentItem(value='T40336799311', est=3, upp... \n",
"Item Price NaN \n",
"Total Tax NaN \n",
"Product Category Code [FrequentItem(value='5.000000', est=232, upper... \n",
"Transaction Day of Week [FrequentItem(value='0.000000', est=909, upper... \n",
"City Code NaN \n",
"Transaction Batch [FrequentItem(value='0.000000', est=909, upper... \n",
"Date of Birth [FrequentItem(value='1981-03-29', est=4, upper... \n",
"Customer ID [FrequentItem(value='C274278', est=4, upper=3,... \n",
"Total Amount NaN \n",
"Product Subcategory [FrequentItem(value='Women', est=133, upper=13... \n",
"Product Subcategory Code [FrequentItem(value='4.000000', est=148, upper... \n",
"\n",
"[18 rows x 24 columns]"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import whylogs as why\n",
"\n",
"# Log the data frame. This equivalent to why.log(retail_daily) and why.log(data=retail_daily)\n",
"results = why.log(pandas=retail_daily)\n",
"\n",
"# Get the Results\n",
"profile = results.profile()\n",
"\n",
"# Head down to Display a Log for explination\n",
"profile.view().to_pandas()"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"## Log Dictionary <a class=\"anchor\" id=\"log_dict\">\n",
"\n",
"Sometimes a quick log is all you need though and don't want to set up a DataFrame. We can log a dictionary as if it were a single row of data. This works best when the values of that dictionary are scalar data, any collection values or nested values will be tracked with only a basic type counter and these entries get mapped to the object count.\n",
"\n",
"Suppose we want to log art prints that are being shown to see what sells best.\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>counts/n</th>\n",
" <th>counts/null</th>\n",
" <th>types/integral</th>\n",
" <th>types/fractional</th>\n",
" <th>types/boolean</th>\n",
" <th>types/string</th>\n",
" <th>types/object</th>\n",
" <th>cardinality/est</th>\n",
" <th>cardinality/upper_1</th>\n",
" <th>cardinality/lower_1</th>\n",
" <th>...</th>\n",
" <th>distribution/n</th>\n",
" <th>distribution/max</th>\n",
" <th>distribution/min</th>\n",
" <th>distribution/q_10</th>\n",
" <th>distribution/q_25</th>\n",
" <th>distribution/median</th>\n",
" <th>distribution/q_75</th>\n",
" <th>distribution/q_90</th>\n",
" <th>ints/max</th>\n",
" <th>ints/min</th>\n",
" </tr>\n",
" <tr>\n",
" <th>column</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>status</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>1.0</td>\n",
" <td>1.00005</td>\n",
" <td>1.0</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>medium</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>price</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1.0</td>\n",
" <td>1.00005</td>\n",
" <td>1.0</td>\n",
" <td>...</td>\n",
" <td>1.0</td>\n",
" <td>58.0</td>\n",
" <td>58.0</td>\n",
" <td>58.0</td>\n",
" <td>58.0</td>\n",
" <td>58.0</td>\n",
" <td>58.0</td>\n",
" <td>58.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>height</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1.0</td>\n",
" <td>1.00005</td>\n",
" <td>1.0</td>\n",
" <td>...</td>\n",
" <td>1.0</td>\n",
" <td>100.0</td>\n",
" <td>100.0</td>\n",
" <td>100.0</td>\n",
" <td>100.0</td>\n",
" <td>100.0</td>\n",
" <td>100.0</td>\n",
" <td>100.0</td>\n",
" <td>100.0</td>\n",
" <td>100.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>length</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1.0</td>\n",
" <td>1.00005</td>\n",
" <td>1.0</td>\n",
" <td>...</td>\n",
" <td>1.0</td>\n",
" <td>1000.0</td>\n",
" <td>1000.0</td>\n",
" <td>1000.0</td>\n",
" <td>1000.0</td>\n",
" <td>1000.0</td>\n",
" <td>1000.0</td>\n",
" <td>1000.0</td>\n",
" <td>1000.0</td>\n",
" <td>1000.0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>5 rows × 24 columns</p>\n",
"</div>"
],
"text/plain": [
" counts/n counts/null types/integral types/fractional \\\n",
"column \n",
"status 1 0 0 0 \n",
"medium 1 0 0 0 \n",
"price 1 0 0 1 \n",
"height 1 0 1 0 \n",
"length 1 0 1 0 \n",
"\n",
" types/boolean types/string types/object cardinality/est \\\n",
"column \n",
"status 0 1 0 1.0 \n",
"medium 0 0 1 NaN \n",
"price 0 0 0 1.0 \n",
"height 0 0 0 1.0 \n",
"length 0 0 0 1.0 \n",
"\n",
" cardinality/upper_1 cardinality/lower_1 ... distribution/n \\\n",
"column ... \n",
"status 1.00005 1.0 ... NaN \n",
"medium NaN NaN ... NaN \n",
"price 1.00005 1.0 ... 1.0 \n",
"height 1.00005 1.0 ... 1.0 \n",
"length 1.00005 1.0 ... 1.0 \n",
"\n",
" distribution/max distribution/min distribution/q_10 \\\n",
"column \n",
"status NaN NaN NaN \n",
"medium NaN NaN NaN \n",
"price 58.0 58.0 58.0 \n",
"height 100.0 100.0 100.0 \n",
"length 1000.0 1000.0 1000.0 \n",
"\n",
" distribution/q_25 distribution/median distribution/q_75 \\\n",
"column \n",
"status NaN NaN NaN \n",
"medium NaN NaN NaN \n",
"price 58.0 58.0 58.0 \n",
"height 100.0 100.0 100.0 \n",
"length 1000.0 1000.0 1000.0 \n",
"\n",
" distribution/q_90 ints/max ints/min \n",
"column \n",
"status NaN NaN NaN \n",
"medium NaN NaN NaN \n",
"price 58.0 NaN NaN \n",
"height 100.0 100.0 100.0 \n",
"length 1000.0 1000.0 1000.0 \n",
"\n",
"[5 rows x 24 columns]"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import whylogs as why\n",
"\n",
"example_data = {\"height\": 100, \"length\": 1000, \"status\": \"sold\", \"price\": 58.00, \"medium\": [\"watercolor\", \"digital\"] }\n",
"\n",
"# Log the dictionary this is equivalent to why.log(example_data)\n",
"dict_results = why.log(row=example_data)\n",
"\n",
"# Retrieve the profile\n",
"profile_from_dict = dict_results.profile()\n",
"\n",
"# Head to Display Logs to explain\n",
"profile_from_dict.view().to_pandas()"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": false
},
"source": [
"## Display the Logs <a class=\"anchor\" id=\"display\"></a>\n",
"There are many ways to display the data! Examples in \"Integrations\", \"WhyLabs\", and \"Use Cases\" showcase how to use a variety of tools to see your data. Also the Notebook_Profile_Visualizer helps you display the profile with a variety of charts.\n",
"\n",
"Your log will from above returns results including the profile. It's this profile that we can view and export as a Pandas DataFrame."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>counts/n</th>\n",
" <th>counts/null</th>\n",
" <th>types/integral</th>\n",
" <th>types/fractional</th>\n",
" <th>types/boolean</th>\n",
" <th>types/string</th>\n",
" <th>types/object</th>\n",
" <th>cardinality/est</th>\n",
" <th>cardinality/upper_1</th>\n",
" <th>cardinality/lower_1</th>\n",
" <th>...</th>\n",
" <th>distribution/min</th>\n",
" <th>distribution/q_10</th>\n",
" <th>distribution/q_25</th>\n",
" <th>distribution/median</th>\n",
" <th>distribution/q_75</th>\n",
" <th>distribution/q_90</th>\n",
" <th>type</th>\n",
" <th>ints/max</th>\n",
" <th>ints/min</th>\n",
" <th>frequent_items/frequent_strings</th>\n",
" </tr>\n",
" <tr>\n",
" <th>column</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>Purchase Canceled</th>\n",
" <td>909</td>\n",
" <td>72</td>\n",
" <td>0</td>\n",
" <td>837</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>2.000000</td>\n",
" <td>2.000100</td>\n",
" <td>2.000000</td>\n",
" <td>...</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Age at Transaction Date</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>25.000001</td>\n",
" <td>25.001250</td>\n",
" <td>25.000000</td>\n",
" <td>...</td>\n",
" <td>19.000</td>\n",
" <td>21.0000</td>\n",
" <td>25.000</td>\n",
" <td>31.0000</td>\n",
" <td>37.0000</td>\n",
" <td>40.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Transaction Week</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1.000000</td>\n",
" <td>1.000050</td>\n",
" <td>1.000000</td>\n",
" <td>...</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>[FrequentItem(value='0.000000', est=909, upper...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Store Type</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>4.000000</td>\n",
" <td>4.000200</td>\n",
" <td>4.000000</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='e-Shop', est=375, upper=3...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Product Category</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>6.000000</td>\n",
" <td>6.000300</td>\n",
" <td>6.000000</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='Books', est=232, upper=23...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Gender</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>2.000000</td>\n",
" <td>2.000100</td>\n",
" <td>2.000000</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='M', est=455, upper=455, l...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Transaction ID</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>904.722898</td>\n",
" <td>916.565225</td>\n",
" <td>893.168643</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='T40336799311', est=3, upp...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Item Price</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>672.542875</td>\n",
" <td>681.346093</td>\n",
" <td>663.953801</td>\n",
" <td>...</td>\n",
" <td>7.100</td>\n",
" <td>18.2000</td>\n",
" <td>43.200</td>\n",
" <td>80.1000</td>\n",
" <td>116.2000</td>\n",
" <td>137.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Total Tax</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>800.975225</td>\n",
" <td>811.459552</td>\n",
" <td>790.745935</td>\n",
" <td>...</td>\n",
" <td>0.861</td>\n",
" <td>4.8825</td>\n",
" <td>10.017</td>\n",
" <td>20.5170</td>\n",
" <td>36.1725</td>\n",
" <td>54.2640</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Product Category Code</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>6.000000</td>\n",
" <td>6.000300</td>\n",
" <td>6.000000</td>\n",
" <td>...</td>\n",
" <td>1.000</td>\n",
" <td>1.0000</td>\n",
" <td>2.000</td>\n",
" <td>4.0000</td>\n",
" <td>5.0000</td>\n",
" <td>6.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>6.0</td>\n",
" <td>1.0</td>\n",
" <td>[FrequentItem(value='5.000000', est=232, upper...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Transaction Day of Week</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1.000000</td>\n",
" <td>1.000050</td>\n",
" <td>1.000000</td>\n",
" <td>...</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>[FrequentItem(value='0.000000', est=909, upper...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>City Code</th>\n",
" <td>909</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>908</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>10.000000</td>\n",
" <td>10.000500</td>\n",
" <td>10.000000</td>\n",
" <td>...</td>\n",
" <td>1.000</td>\n",
" <td>1.0000</td>\n",
" <td>3.000</td>\n",
" <td>5.0000</td>\n",
" <td>8.0000</td>\n",
" <td>10.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Transaction Batch</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1.000000</td>\n",
" <td>1.000050</td>\n",
" <td>1.000000</td>\n",
" <td>...</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>0.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>[FrequentItem(value='0.000000', est=909, upper...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Date of Birth</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>801.978113</td>\n",
" <td>812.475567</td>\n",
" <td>791.736016</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='1981-03-29', est=4, upper...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Customer ID</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>847.420398</td>\n",
" <td>858.512667</td>\n",
" <td>836.597955</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='C274278', est=4, upper=3,...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Total Amount</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>842.098548</td>\n",
" <td>853.121157</td>\n",
" <td>831.344071</td>\n",
" <td>...</td>\n",
" <td>-767.975</td>\n",
" <td>25.3045</td>\n",
" <td>82.433</td>\n",
" <td>188.4025</td>\n",
" <td>358.6830</td>\n",
" <td>555.2625</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Product Subcategory</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>18.000001</td>\n",
" <td>18.000899</td>\n",
" <td>18.000000</td>\n",
" <td>...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[FrequentItem(value='Women', est=133, upper=13...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>Product Subcategory Code</th>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>909</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>12.000000</td>\n",
" <td>12.000599</td>\n",
" <td>12.000000</td>\n",
" <td>...</td>\n",
" <td>1.000</td>\n",
" <td>1.0000</td>\n",
" <td>3.000</td>\n",
" <td>5.0000</td>\n",
" <td>10.0000</td>\n",
" <td>11.0000</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>12.0</td>\n",
" <td>1.0</td>\n",
" <td>[FrequentItem(value='4.000000', est=148, upper...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>18 rows × 24 columns</p>\n",
"</div>"
],
"text/plain": [
" counts/n counts/null types/integral \\\n",
"column \n",
"Purchase Canceled 909 72 0 \n",
"Age at Transaction Date 909 0 0 \n",
"Transaction Week 909 0 909 \n",
"Store Type 909 0 0 \n",
"Product Category 909 0 0 \n",
"Gender 909 0 0 \n",
"Transaction ID 909 0 0 \n",
"Item Price 909 0 0 \n",
"Total Tax 909 0 0 \n",
"Product Category Code 909 0 909 \n",
"Transaction Day of Week 909 0 909 \n",
"City Code 909 1 0 \n",
"Transaction Batch 909 0 909 \n",
"Date of Birth 909 0 0 \n",
"Customer ID 909 0 0 \n",
"Total Amount 909 0 0 \n",
"Product Subcategory 909 0 0 \n",
"Product Subcategory Code 909 0 909 \n",
"\n",
" types/fractional types/boolean types/string \\\n",
"column \n",
"Purchase Canceled 837 0 0 \n",
"Age at Transaction Date 909 0 0 \n",
"Transaction Week 0 0 0 \n",
"Store Type 0 0 909 \n",
"Product Category 0 0 909 \n",
"Gender 0 0 909 \n",
"Transaction ID 0 0 909 \n",
"Item Price 909 0 0 \n",
"Total Tax 909 0 0 \n",
"Product Category Code 0 0 0 \n",
"Transaction Day of Week 0 0 0 \n",
"City Code 908 0 0 \n",
"Transaction Batch 0 0 0 \n",
"Date of Birth 0 0 909 \n",
"Customer ID 0 0 909 \n",
"Total Amount 909 0 0 \n",
"Product Subcategory 0 0 909 \n",
"Product Subcategory Code 0 0 0 \n",
"\n",
" types/object cardinality/est cardinality/upper_1 \\\n",
"column \n",
"Purchase Canceled 0 2.000000 2.000100 \n",
"Age at Transaction Date 0 25.000001 25.001250 \n",
"Transaction Week 0 1.000000 1.000050 \n",
"Store Type 0 4.000000 4.000200 \n",
"Product Category 0 6.000000 6.000300 \n",
"Gender 0 2.000000 2.000100 \n",
"Transaction ID 0 904.722898 916.565225 \n",
"Item Price 0 672.542875 681.346093 \n",
"Total Tax 0 800.975225 811.459552 \n",
"Product Category Code 0 6.000000 6.000300 \n",
"Transaction Day of Week 0 1.000000 1.000050 \n",
"City Code 0 10.000000 10.000500 \n",
"Transaction Batch 0 1.000000 1.000050 \n",
"Date of Birth 0 801.978113 812.475567 \n",
"Customer ID 0 847.420398 858.512667 \n",
"Total Amount 0 842.098548 853.121157 \n",
"Product Subcategory 0 18.000001 18.000899 \n",
"Product Subcategory Code 0 12.000000 12.000599 \n",
"\n",
" cardinality/lower_1 ... distribution/min \\\n",
"column ... \n",
"Purchase Canceled 2.000000 ... 0.000 \n",
"Age at Transaction Date 25.000000 ... 19.000 \n",
"Transaction Week 1.000000 ... 0.000 \n",
"Store Type 4.000000 ... NaN \n",
"Product Category 6.000000 ... NaN \n",
"Gender 2.000000 ... NaN \n",
"Transaction ID 893.168643 ... NaN \n",
"Item Price 663.953801 ... 7.100 \n",
"Total Tax 790.745935 ... 0.861 \n",
"Product Category Code 6.000000 ... 1.000 \n",
"Transaction Day of Week 1.000000 ... 0.000 \n",
"City Code 10.000000 ... 1.000 \n",
"Transaction Batch 1.000000 ... 0.000 \n",
"Date of Birth 791.736016 ... NaN \n",
"Customer ID 836.597955 ... NaN \n",
"Total Amount 831.344071 ... -767.975 \n",
"Product Subcategory 18.000000 ... NaN \n",
"Product Subcategory Code 12.000000 ... 1.000 \n",
"\n",
" distribution/q_10 distribution/q_25 \\\n",
"column \n",
"Purchase Canceled 0.0000 0.000 \n",
"Age at Transaction Date 21.0000 25.000 \n",
"Transaction Week 0.0000 0.000 \n",
"Store Type NaN NaN \n",
"Product Category NaN NaN \n",
"Gender NaN NaN \n",
"Transaction ID NaN NaN \n",
"Item Price 18.2000 43.200 \n",
"Total Tax 4.8825 10.017 \n",
"Product Category Code 1.0000 2.000 \n",
"Transaction Day of Week 0.0000 0.000 \n",
"City Code 1.0000 3.000 \n",
"Transaction Batch 0.0000 0.000 \n",
"Date of Birth NaN NaN \n",
"Customer ID NaN NaN \n",
"Total Amount 25.3045 82.433 \n",
"Product Subcategory NaN NaN \n",
"Product Subcategory Code 1.0000 3.000 \n",
"\n",
" distribution/median distribution/q_75 \\\n",
"column \n",
"Purchase Canceled 0.0000 0.0000 \n",
"Age at Transaction Date 31.0000 37.0000 \n",
"Transaction Week 0.0000 0.0000 \n",
"Store Type NaN NaN \n",
"Product Category NaN NaN \n",
"Gender NaN NaN \n",
"Transaction ID NaN NaN \n",
"Item Price 80.1000 116.2000 \n",
"Total Tax 20.5170 36.1725 \n",
"Product Category Code 4.0000 5.0000 \n",
"Transaction Day of Week 0.0000 0.0000 \n",
"City Code 5.0000 8.0000 \n",
"Transaction Batch 0.0000 0.0000 \n",
"Date of Birth NaN NaN \n",
"Customer ID NaN NaN \n",
"Total Amount 188.4025 358.6830 \n",
"Product Subcategory NaN NaN \n",
"Product Subcategory Code 5.0000 10.0000 \n",
"\n",
" distribution/q_90 type ints/max \\\n",
"column \n",
"Purchase Canceled 0.0000 SummaryType.COLUMN NaN \n",
"Age at Transaction Date 40.0000 SummaryType.COLUMN NaN \n",
"Transaction Week 0.0000 SummaryType.COLUMN 0.0 \n",
"Store Type NaN SummaryType.COLUMN NaN \n",
"Product Category NaN SummaryType.COLUMN NaN \n",
"Gender NaN SummaryType.COLUMN NaN \n",
"Transaction ID NaN SummaryType.COLUMN NaN \n",
"Item Price 137.0000 SummaryType.COLUMN NaN \n",
"Total Tax 54.2640 SummaryType.COLUMN NaN \n",
"Product Category Code 6.0000 SummaryType.COLUMN 6.0 \n",
"Transaction Day of Week 0.0000 SummaryType.COLUMN 0.0 \n",
"City Code 10.0000 SummaryType.COLUMN NaN \n",
"Transaction Batch 0.0000 SummaryType.COLUMN 0.0 \n",
"Date of Birth NaN SummaryType.COLUMN NaN \n",
"Customer ID NaN SummaryType.COLUMN NaN \n",
"Total Amount 555.2625 SummaryType.COLUMN NaN \n",
"Product Subcategory NaN SummaryType.COLUMN NaN \n",
"Product Subcategory Code 11.0000 SummaryType.COLUMN 12.0 \n",
"\n",
" ints/min \\\n",
"column \n",
"Purchase Canceled NaN \n",
"Age at Transaction Date NaN \n",
"Transaction Week 0.0 \n",
"Store Type NaN \n",
"Product Category NaN \n",
"Gender NaN \n",
"Transaction ID NaN \n",
"Item Price NaN \n",
"Total Tax NaN \n",
"Product Category Code 1.0 \n",
"Transaction Day of Week 0.0 \n",
"City Code NaN \n",
"Transaction Batch 0.0 \n",
"Date of Birth NaN \n",
"Customer ID NaN \n",
"Total Amount NaN \n",
"Product Subcategory NaN \n",
"Product Subcategory Code 1.0 \n",
"\n",
" frequent_items/frequent_strings \n",
"column \n",
"Purchase Canceled NaN \n",
"Age at Transaction Date NaN \n",
"Transaction Week [FrequentItem(value='0.000000', est=909, upper... \n",
"Store Type [FrequentItem(value='e-Shop', est=375, upper=3... \n",
"Product Category [FrequentItem(value='Books', est=232, upper=23... \n",
"Gender [FrequentItem(value='M', est=455, upper=455, l... \n",
"Transaction ID [FrequentItem(value='T40336799311', est=3, upp... \n",
"Item Price NaN \n",
"Total Tax NaN \n",
"Product Category Code [FrequentItem(value='5.000000', est=232, upper... \n",
"Transaction Day of Week [FrequentItem(value='0.000000', est=909, upper... \n",
"City Code NaN \n",
"Transaction Batch [FrequentItem(value='0.000000', est=909, upper... \n",
"Date of Birth [FrequentItem(value='1981-03-29', est=4, upper... \n",
"Customer ID [FrequentItem(value='C274278', est=4, upper=3,... \n",
"Total Amount NaN \n",
"Product Subcategory [FrequentItem(value='Women', est=133, upper=13... \n",
"Product Subcategory Code [FrequentItem(value='4.000000', est=148, upper... \n",
"\n",
"[18 rows x 24 columns]"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Note run any of the examples above to get the results for this block\n",
"\n",
"#grab profile from result set\n",
"profile = results.profile()\n",
"\n",
"#grab a 'view' of the profile for inspection\n",
"prof_view = profile.view()\n",
"\n",
"#inspect profile as a Pandas DataFrame\n",
"prof_df = prof_view.to_pandas()\n",
"prof_df"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.9.5 ('base')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.5"
},
"vscode": {
"interpreter": {
"hash": "8148c804a5838694570acf40aa3269caeebb6c584d51452dd558e946dfc16d39"
}
}
},
"nbformat": 4,
"nbformat_minor": 0
}