python/examples/integrations/writers/Writing_Reference_Profiles_to_WhyLabs.ipynb
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
">### 🚩 *Create a free WhyLabs account to get more value out of whylogs!*<br> \n",
">*Did you know you can store, visualize, and monitor whylogs profiles with the [WhyLabs Observability Platform](https://whylabs.ai/whylogs-free-signup?utm_source=whylogs-Github&utm_medium=whylogs-example&utm_campaign=Writing_Reference_Profiles_to_WhyLabs)? Sign up for a [free WhyLabs account](https://whylabs.ai/whylogs-free-signup?utm_source=whylogs-Github&utm_medium=whylogs-example&utm_campaign=Writing_Reference_Profiles_to_WhyLabs) to leverage the power of whylogs and WhyLabs together!*"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Writing Reference Profiles to WhyLabs"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"[](https://colab.research.google.com/github/whylabs/whylogs/blob/mainline/python/examples/integrations/writers/Writing_Reference_Profiles_to_WhyLabs.ipynb)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"When monitoring your data, in many cases you'll be interested in comparing data from your production pipeline with a reference, or baseline, profile. This is helpful when inspecting for data drift, or assessing the quality of your data in general.\n",
"\n",
"In this example, we'll show how to send a profile logged with whylogs to your monitoring dashboard at WhyLabs Platform as a Reference Profile. When uploading a Reference Profile, you'll be able to use it for visualization and comparison purposes on your monitoring dashboard.\n",
"\n",
"> If you want to log your profiles as regular profiles (_Batch Profiles_), as opposed to _Reference Profiles_, please check the [Writing to WhyLabs](https://whylogs.readthedocs.io/en/stable/examples/integrations/writers/Writing_to_WhyLabs.html) example.\n",
"\n",
"We will:\n",
"\n",
"- Define environment variables with the appropriate Credentials and IDs\n",
"- Log data into a profile\n",
"- Use the WhyLabs Writer to send the profile as a Reference Profile to your Project at WhyLabs"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Installing whylogs\n",
"\n",
"First, let's install __whylogs__. Since we want to write to WhyLabs, we'll also install the __whylabs__ extra.\n",
"\n",
"If you don't have it installed already, uncomment the line below:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: whylogs in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (1.3.1)\n",
"Requirement already satisfied: platformdirs<4.0.0,>=3.5.0 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from whylogs) (3.10.0)\n",
"Requirement already satisfied: protobuf>=3.19.4 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from whylogs) (4.24.2)\n",
"Requirement already satisfied: requests<3.0,>=2.27 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from whylogs) (2.31.0)\n",
"Requirement already satisfied: types-requests<3.0.0.0,>=2.30.0.0 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from whylogs) (2.31.0.2)\n",
"Requirement already satisfied: typing-extensions>=3.10 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from whylogs) (4.7.1)\n",
"Requirement already satisfied: whylabs-client<0.6.0,>=0.5.5 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from whylogs) (0.5.5)\n",
"Requirement already satisfied: whylogs-sketching>=3.4.1.dev3 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from whylogs) (3.4.1.dev3)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from requests<3.0,>=2.27->whylogs) (3.2.0)\n",
"Requirement already satisfied: idna<4,>=2.5 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from requests<3.0,>=2.27->whylogs) (3.4)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from requests<3.0,>=2.27->whylogs) (1.26.16)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from requests<3.0,>=2.27->whylogs) (2023.7.22)\n",
"Requirement already satisfied: types-urllib3 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from types-requests<3.0.0.0,>=2.30.0.0->whylogs) (1.26.25.14)\n",
"Requirement already satisfied: python-dateutil in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from whylabs-client<0.6.0,>=0.5.5->whylogs) (2.8.2)\n",
"Requirement already satisfied: six>=1.5 in /home/anthony/workspace/whylogs/python/.venv/lib/python3.9/site-packages (from python-dateutil->whylabs-client<0.6.0,>=0.5.5->whylogs) (1.16.0)\n",
"\u001b[33mDEPRECATION: feast 0.22.4 has a non-standard dependency specifier googleapis-common-protos<2,>=1.52.*. pip 23.3 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of feast or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063\u001b[0m\u001b[33m\n",
"\u001b[0m\u001b[33mDEPRECATION: feast 0.22.4 has a non-standard dependency specifier PyYAML<7,>=5.4.*. pip 23.3 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of feast or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063\u001b[0m\u001b[33m\n",
"\u001b[0m\u001b[33mDEPRECATION: feast 0.22.4 has a non-standard dependency specifier dask<2022.02.0,>=2021.*. pip 23.3 will enforce this behaviour change. A possible replacement is to upgrade to a newer version of feast or contact the author to suggest that they release a version with a conforming dependency specifiers. Discussion can be found at https://github.com/pypa/pip/issues/12063\u001b[0m\u001b[33m\n",
"\u001b[0mNote: you may need to restart the kernel to use updated packages.\n"
]
}
],
"source": [
"# Note: you may need to restart the kernel to use updated packages.\n",
"%pip install whylogs"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## ✔️ Setting the Environment Variables"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"In order to send our profile to WhyLabs, let's first set up an account. You can skip this if you already have an account and a model set up.\n",
"\n",
"We will need three pieces of information:\n",
"\n",
"- API token\n",
"- Organization ID\n",
"- Dataset ID (or model-id)\n",
"\n",
"Go to https://whylabs.ai/free and grab a free account. You can follow along with the examples if you wish, but if you’re interested in only following this demonstration, you can go ahead and skip the quick start instructions.\n",
"\n",
"After that, you’ll be prompted to create an API token. Once you create it, copy and store it locally. The second important information here is your org ID. Take note of it as well. After you get your API Token and Org ID, you can go to https://hub.whylabsapp.com/models to see your projects dashboard. You can create a new project and take note of it's ID (if it's a model project it will look like `model-xxxx`)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import whylogs as why\n",
"\n",
"why.init()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Fetching the Data\n",
"\n",
"For demonstration, let's use data for transactions from a small retail business:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Transaction ID</th>\n",
" <th>Customer ID</th>\n",
" <th>Quantity</th>\n",
" <th>Item Price</th>\n",
" <th>Total Tax</th>\n",
" <th>Total Amount</th>\n",
" <th>Store Type</th>\n",
" <th>Product Category</th>\n",
" <th>Product Subcategory</th>\n",
" <th>Gender</th>\n",
" <th>Transaction Type</th>\n",
" <th>Age</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>T14259136777</td>\n",
" <td>C274477</td>\n",
" <td>1</td>\n",
" <td>148.9</td>\n",
" <td>15.6345</td>\n",
" <td>164.5345</td>\n",
" <td>TeleShop</td>\n",
" <td>Electronics</td>\n",
" <td>Audio and video</td>\n",
" <td>F</td>\n",
" <td>Purchase</td>\n",
" <td>37.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>T7313351894</td>\n",
" <td>C267568</td>\n",
" <td>4</td>\n",
" <td>48.1</td>\n",
" <td>20.2020</td>\n",
" <td>212.6020</td>\n",
" <td>Flagship store</td>\n",
" <td>Home and kitchen</td>\n",
" <td>Furnishing</td>\n",
" <td>M</td>\n",
" <td>Purchase</td>\n",
" <td>25.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>T37745642681</td>\n",
" <td>C267098</td>\n",
" <td>1</td>\n",
" <td>10.9</td>\n",
" <td>1.1445</td>\n",
" <td>12.0445</td>\n",
" <td>Flagship store</td>\n",
" <td>Footwear</td>\n",
" <td>Mens</td>\n",
" <td>F</td>\n",
" <td>Purchase</td>\n",
" <td>42.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>T13861409908</td>\n",
" <td>C271608</td>\n",
" <td>2</td>\n",
" <td>135.2</td>\n",
" <td>28.3920</td>\n",
" <td>298.7920</td>\n",
" <td>MBR</td>\n",
" <td>Footwear</td>\n",
" <td>Mens</td>\n",
" <td>F</td>\n",
" <td>Purchase</td>\n",
" <td>43.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>T58956348529</td>\n",
" <td>C272484</td>\n",
" <td>4</td>\n",
" <td>144.3</td>\n",
" <td>60.6060</td>\n",
" <td>637.8060</td>\n",
" <td>TeleShop</td>\n",
" <td>Clothing</td>\n",
" <td>Mens</td>\n",
" <td>F</td>\n",
" <td>Purchase</td>\n",
" <td>39.0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Transaction ID Customer ID Quantity Item Price Total Tax Total Amount \\\n",
"0 T14259136777 C274477 1 148.9 15.6345 164.5345 \n",
"1 T7313351894 C267568 4 48.1 20.2020 212.6020 \n",
"2 T37745642681 C267098 1 10.9 1.1445 12.0445 \n",
"3 T13861409908 C271608 2 135.2 28.3920 298.7920 \n",
"4 T58956348529 C272484 4 144.3 60.6060 637.8060 \n",
"\n",
" Store Type Product Category Product Subcategory Gender \\\n",
"0 TeleShop Electronics Audio and video F \n",
"1 Flagship store Home and kitchen Furnishing M \n",
"2 Flagship store Footwear Mens F \n",
"3 MBR Footwear Mens F \n",
"4 TeleShop Clothing Mens F \n",
"\n",
" Transaction Type Age \n",
"0 Purchase 37.0 \n",
"1 Purchase 25.0 \n",
"2 Purchase 42.0 \n",
"3 Purchase 43.0 \n",
"4 Purchase 39.0 "
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"\n",
"csv_url = \"https://whylabs-public.s3.us-west-2.amazonaws.com/datasets/tour/current.csv\"\n",
"df = pd.read_csv(csv_url)\n",
"\n",
"df.head()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## 📊 Profiling the Data\n",
"\n",
"Let's profile the data with whylogs:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import whylogs as why\n",
"from datetime import datetime, timezone\n",
"current_date = datetime.now(timezone.utc)\n",
"profile = why.log(df, dataset_timestamp=current_date, name=\"tour\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We're setting the profile's dataset timestamp as the current datetime. If this is not set, the Writer would simply assign the current datetime automatically to the profile."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"In this case, We named the refrence profile `tour`, which means we can find it in the profile page under that name. We could also name it `\"\"` (empty string) which tells WhyLabs to generate a name for it based on the timestamp, like `ref-2022-08-16T17:53:49.041`"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## 🔍 A Look on the Other Side\n",
"\n",
"Now, check your dashboard to verify everything went ok. At the __Profile__ tab, you should see something like this:"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"In the image above, we're comparing both reference profiles sent previously. Usually, we'd be interested in comparing a reference profile with a batch profile obtained in the production pipeline, which is, of course, also possible."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.9.13 ('.venv': poetry)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
},
"orig_nbformat": 4,
"vscode": {
"interpreter": {
"hash": "0f484380554f045e8316d9ef136659363ef199c84a7347221e49b73e46486d36"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}