python/examples/integrations/Feature_Stores_and_whylogs.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
">### 🚩 *Create a free WhyLabs account to get more value out of whylogs!*<br> \n",
">*Did you know you can store, visualize, and monitor whylogs profiles with the [WhyLabs Observability Platform](https://whylabs.ai/whylogs-free-signup?utm_source=whylogs-Github&utm_medium=whylogs-example&utm_campaign=Feature_Stores_and_whylogs)? Sign up for a [free WhyLabs account](https://whylabs.ai/whylogs-free-signup?utm_source=whylogs-Github&utm_medium=whylogs-example&utm_campaign=Feature_Stores_and_whylogs) to leverage the power of whylogs and WhyLabs together!*"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "W5HhG0NE8Gkf"
},
"source": [
"# Logging data from Feature Stores with Feast and whylogs"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[](https://colab.research.google.com/github/whylabs/whylogs/blob/mainline/python/examples/integrations/Feature_Stores_and_whylogs.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> This is a `whylogs v1` example. For the analog example in `v0`, please refer to [this example](https://github.com/whylabs/whylogs/blob/maintenance/0.7.x/examples/feast_whylogs_example/feast_whylogs.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "oFQUD_anA1ie"
},
"source": [
"## Context\n",
"\n",
"In this walkthrough, we'll see how you can use Feast and whylogs together at different parts of your ML pipeline - We'll use Feast to set up an online feature store, and then use it to enrich our serving data with additional features. After assembling our feature vector, we'll proceed to log it with whylogs. As the requests for prediction arrives, the logged input features will be statistically profiled. We will explore these profiles to see what kind of insights we can have.\n",
"\n",
"\n",
"\n",
"To do so, we'll use a sample dataset of daily taxi rides in NYC, extracted from [here](https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page). Our final goal could be a prediction requested at the start of a given ride. This prediction could be whether the customer will give a high tip to the driver, or maybe whether the customer will give him a good review. As an input to the prediction model, in addition to the ride information (like number of passengers, day of the week, or trip distance), we might be interested in enriching our feature vector with information about the driver, like the driver's average speed, average rating or avg trips in the last 24 hours, with the hopes of improving the model's performances.\n",
"\n",
"![alt text](https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/feast_integration/images/context.jpg \"Title\")\n",
"\n",
"The info about the specific ride will be known at inference time. However, the driver statistics might be available to us in a different data source, updated at specific time intervals. We will join these information to assemble a single feature vector by using Feast to set up an online feature store. Feast will materialize the features into the online store from a data source file. This data source will have driver statistic's, according the each driver's ID updated in an hourly basis.\n",
"\n",
"We will simulate a production pipeline, where requests for predictions will be made at different timestamps. We'll then log the feature vectors for each request into daily profiles for a period of 7 days. We'll then see how we can compare the obtained profiles for possible data issues or drifts we might have between days."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Qw5QyJ3qBi5f"
},
"source": [
"## Changes in Data\n",
"\n",
"Let's consider some scenarios in which logging and visualizing features would be helpful.\n",
"\n",
"### Data Freshness\n",
"\n",
"In this example, we have updated information about drivers in an hourly basis. Let's simulate a scenario in which this frequency gets affected by some reason, and for a particular period we have new information accessible only in 2-hour cycles.\n",
"\n",
"### Changes in Customer Behavior\n",
"\n",
"Let's consider a scenario where people's behavior changes: maybe people are riding less. For example, when covid started, the number of rides certainly plummeted. We could also have a change in the criterias people use to rate a driver. For example, now the given rates, or reviews, for each driver could be affected by specific services provided, like the presence of alcohol and/or physical barriers to ensure social distancing. "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bz21zNiWBtcO"
},
"source": [
"## The Feature Repository"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NoHSMmLqGGIm"
},
"source": [
"First of all, let's install the required packages for this tutorial:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Note: you may need to restart the kernel to use updated packages.\n",
"%pip install --upgrade pip -qq\n",
"%pip install feast==0.22.4 -qq\n",
"%pip install Pygments -qq\n",
"%pip install 'whylogs[viz]' -U"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1wsv4K6IBziV"
},
"source": [
"### Boilerplate - Registering feature definitions and deploying your feature store"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "K8RgLWexB73l"
},
"source": [
"In order to deploy our feature store, we need to create a feature repository. In [Feast's quickstart example](https://docs.feast.dev/getting-started/quickstart), this is traditionally done with a `feast init` command. This example is based on the quickstart example, but with some changes in the python and configuration files.\n",
"\n",
"For this reason, let's quickly create a folder with the required files to create a feature repository adapted to our use case."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5LfOoBnaC1gw"
},
"outputs": [],
"source": [
"%%sh\n",
"mkdir feature_repo\n",
"mkdir feature_repo/data\n",
"mkdir feature_repo/whylogs_output\n",
"touch feature_repo/__init__.py"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Writing our feature definition in the `example.py` inside our `feature_repo` folder:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "hXp3XAesD3xl",
"outputId": "4f8c9d49-fa74-4220-cb9b-caadad338561"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Overwriting feature_repo/example.py\n"
]
}
],
"source": [
"%%writefile feature_repo/example.py\n",
"# This is an example feature definition file\n",
"\n",
"from datetime import timedelta\n",
"\n",
"from feast import Entity, FeatureView, Field, FeatureService, FileSource, ValueType\n",
"from feast.types import Float32, Int64\n",
"\n",
"# Read data from parquet files. Parquet is convenient for local development mode. For\n",
"# production, you can use your favorite DWH, such as BigQuery. See Feast documentation\n",
"# for more info.\n",
"driver_hourly_stats = FileSource(\n",
" path=\"data/driver_stats.parquet\",\n",
" timestamp_field=\"event_timestamp\",\n",
" created_timestamp_column=\"created\",\n",
")\n",
"\n",
"# Define an entity for the driver. You can think of entity as a primary key used to\n",
"# fetch features.\n",
"# Entity has a name used for later reference (in a feature view, eg)\n",
"# and join_key to identify physical field name used in storages\n",
"driver = Entity(name=\"driver\", value_type=ValueType.INT64, join_keys=[\"driver_id\"], description=\"driver id\",)\n",
"\n",
"# Our parquet files contain sample data that includes a driver_id column, timestamps and\n",
"# three feature column. Here we define a Feature View that will allow us to serve this\n",
"# data to our model online.\n",
"driver_hourly_stats_view = FeatureView(\n",
" name=\"driver_hourly_stats\",\n",
" entities=[\"driver\"], # reference entity by name\n",
" ttl=timedelta(seconds=86400 * 1),\n",
" schema=[\n",
" Field(name=\"rate_1m\", dtype=Int64),\n",
" Field(name=\"avg_daily_trips\", dtype=Int64),\n",
" Field(name=\"avg_speed\", dtype=Float32),\n",
"\n",
" ],\n",
" online=True,\n",
" source=driver_hourly_stats,\n",
" tags={},\n",
")\n",
"\n",
"driver_stats_fs = FeatureService(\n",
" name=\"driver_activity\",\n",
" features=[driver_hourly_stats_view]\n",
")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Writing the `feature_store.yaml` configuration file:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Iiq0DUK2EBde",
"outputId": "5133b995-8d44-40fd-8795-4af9cd1e4596"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Overwriting feature_repo/feature_store.yaml\n"
]
}
],
"source": [
"%%writefile feature_repo/feature_store.yaml\n",
"project: feature_repo\n",
"registry: data/registry.db\n",
"provider: local\n",
"online_store:\n",
" path: data/online_store.db"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "l5JrYDD_FDXK"
},
"source": [
"### Downloading the Data Source"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "oq-X_vAzFT4E"
},
"source": [
"Let's first navigate to our feature repository folder:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "cTb0fwR9Fdad",
"outputId": "de993da7-011e-44e1-9836-95115f799bf9"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/mnt/c/Users/felip/Documents/Projects-WhyLabs/whylogs2/python/examples/integrations/feature_repo\n"
]
}
],
"source": [
"%cd feature_repo"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o7ltooeMFg6d"
},
"source": [
"Make sure you're on the right folder. You should see an empty data folder (we'll populate it with our data source later), the `example.py` python script, which contains our feature definitions, and the `feature_store.yaml` configuration file."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "3ueZRWSyFmgY",
"outputId": "1e1429e8-3591-4e7e-8917-1cd2c29f721f"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
".:\n",
"\u001b[0m\u001b[01;32m__init__.py\u001b[0m* \u001b[34;42mdata\u001b[0m/ \u001b[01;32mexample.py\u001b[0m* \u001b[01;32mfeature_store.yaml\u001b[0m* \u001b[34;42mwhylogs_output\u001b[0m/\n",
"\n",
"./data:\n",
"\u001b[01;32mdriver_stats.parquet\u001b[0m* \u001b[01;32mregistry.db\u001b[0m*\n",
"\n",
"./whylogs_output:\n"
]
}
],
"source": [
"%ls -R"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ci3PSbC5Fred"
},
"source": [
"Now, let's download our data source and store it locally in our feature repository:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'0.22.4'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import feast\n",
"\n",
"feast.__version__"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "VUTlby8YFvAH",
"outputId": "056268fa-6815-428a-c016-b6658f6417bc"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Loading data from https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/feast_integration/driver_stats.parquet\n",
"Saving file source locally\n"
]
}
],
"source": [
"import pandas as pd\n",
"path = f\"https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/feast_integration/driver_stats.parquet\"\n",
"print(f\"Loading data from {path}\")\n",
"driver_stats = pd.read_parquet(path)\n",
"print(f\"Saving file source locally\")\n",
"\n",
"driver_stats.to_parquet(\"data/driver_stats.parquet\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "W8fsPXZGFy6v"
},
"source": [
"In the data source, we have driver's statistics on an hourly basis, such as the average trips done on the last 24 hours, average rating on the last month and average driving speed. You can see more information on how this data was created at the end of this notebook, in the Appendix."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 206
},
"id": "C7CrqYIMF1bG",
"outputId": "fc588944-ce4d-41fa-b2e2-fee0a78b63a7"
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>index</th>\n",
" <th>event_timestamp</th>\n",
" <th>driver_id</th>\n",
" <th>created</th>\n",
" <th>avg_daily_trips</th>\n",
" <th>rate_1m</th>\n",
" <th>avg_speed</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>0.0</td>\n",
" <td>2020-02-10 00:00:00</td>\n",
" <td>1001</td>\n",
" <td>2022-02-16 16:17:56.446774</td>\n",
" <td>25</td>\n",
" <td>3</td>\n",
" <td>16.87</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>0.0</td>\n",
" <td>2020-02-10 00:00:00</td>\n",
" <td>1002</td>\n",
" <td>2022-02-16 16:17:56.446774</td>\n",
" <td>35</td>\n",
" <td>1</td>\n",
" <td>20.21</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>1.0</td>\n",
" <td>2020-02-10 01:00:00</td>\n",
" <td>1001</td>\n",
" <td>2022-02-16 16:17:56.446774</td>\n",
" <td>19</td>\n",
" <td>4</td>\n",
" <td>20.77</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1.0</td>\n",
" <td>2020-02-10 01:00:00</td>\n",
" <td>1002</td>\n",
" <td>2022-02-16 16:17:56.446774</td>\n",
" <td>29</td>\n",
" <td>3</td>\n",
" <td>19.20</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>2.0</td>\n",
" <td>2020-02-10 02:00:00</td>\n",
" <td>1001</td>\n",
" <td>2022-02-16 16:17:56.446774</td>\n",
" <td>31</td>\n",
" <td>3</td>\n",
" <td>17.41</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" index event_timestamp driver_id created \\\n",
"0 0.0 2020-02-10 00:00:00 1001 2022-02-16 16:17:56.446774 \n",
"1 0.0 2020-02-10 00:00:00 1002 2022-02-16 16:17:56.446774 \n",
"2 1.0 2020-02-10 01:00:00 1001 2022-02-16 16:17:56.446774 \n",
"3 1.0 2020-02-10 01:00:00 1002 2022-02-16 16:17:56.446774 \n",
"4 2.0 2020-02-10 02:00:00 1001 2022-02-16 16:17:56.446774 \n",
"\n",
" avg_daily_trips rate_1m avg_speed \n",
"0 25 3 16.87 \n",
"1 35 1 20.21 \n",
"2 19 4 20.77 \n",
"3 29 3 19.20 \n",
"4 31 3 17.41 "
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"driver_stats.head()"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"index float64\n",
"event_timestamp datetime64[ns]\n",
"driver_id int64\n",
"created datetime64[ns]\n",
"avg_daily_trips int64\n",
"rate_1m int64\n",
"avg_speed float64\n",
"dtype: object"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"driver_stats.dtypes"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BzjZA_3mGdiO"
},
"source": [
"### Deploying the Feature Store"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KvjqWLdMF9Uf"
},
"source": [
"Now, we will scan the python files in our feature repository for feature views/entity definitions, register the objects and deploy the infrastructure with the `feast apply command`."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "eum0PevxG4Oe",
"outputId": "a9aa75eb-1af0-40df-c85f-2f759e09f529"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/mnt/c/Users/felip/Documents/Projects-WhyLabs/whylogs2/python/.venv/lib/python3.8/site-packages/feast/entity.py:110: DeprecationWarning: The `value_type` parameter is being deprecated. Instead, the type of an entity should be specified as a Field in the schema of a feature view. Feast 0.24 and onwards will not support the `value_type` parameter. The `entities` parameter of feature views should also be changed to a List[Entity] instead of a List[str]; if this is not done, entity columns will be mistakenly interpreted as feature columns.\n",
" warnings.warn(\n",
"/mnt/c/Users/felip/Documents/Projects-WhyLabs/whylogs2/python/.venv/lib/python3.8/site-packages/feast/feature_view.py:180: DeprecationWarning: The `entities` parameter should be a list of `Entity` objects. Feast 0.24 and onwards will not support passing in a list of strings to define entities.\n",
" warnings.warn(\n",
"Created entity \u001b[1m\u001b[32mdriver\u001b[0m\n",
"Created feature view \u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m\n",
"Created feature service \u001b[1m\u001b[32mdriver_activity\u001b[0m\n",
"\n",
"Created sqlite table \u001b[1m\u001b[32mfeature_repo_driver_hourly_stats\u001b[0m\n",
"\n"
]
}
],
"source": [
"!feast apply"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "375FJo5AXD4a"
},
"source": [
"Let's also load our rides dataframe. In it we, have features about rides made during 10-Feb to 16-Feb (2020), such as the number of passengers, trip distance and pickup date and time. "
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 223
},
"id": "EFceJHojXGgZ",
"outputId": "1e0b0028-4058-4bd9-90ee-a5972f68c04c"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Loading data from https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/nyc_taxi_rides_feb_2020_changed.parquet\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>pickup_weekday</th>\n",
" <th>passenger_count</th>\n",
" <th>trip_distance</th>\n",
" <th>PULocationID</th>\n",
" <th>tpep_pickup_datetime</th>\n",
" <th>pickup_date</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>225897</th>\n",
" <td>0</td>\n",
" <td>1.0</td>\n",
" <td>1.20</td>\n",
" <td>249</td>\n",
" <td>2020-02-10 00:23:21</td>\n",
" <td>2020-02-10</td>\n",
" </tr>\n",
" <tr>\n",
" <th>108301</th>\n",
" <td>0</td>\n",
" <td>5.0</td>\n",
" <td>19.03</td>\n",
" <td>132</td>\n",
" <td>2020-02-10 01:19:01</td>\n",
" <td>2020-02-10</td>\n",
" </tr>\n",
" <tr>\n",
" <th>196729</th>\n",
" <td>0</td>\n",
" <td>6.0</td>\n",
" <td>0.38</td>\n",
" <td>68</td>\n",
" <td>2020-02-10 01:29:23</td>\n",
" <td>2020-02-10</td>\n",
" </tr>\n",
" <tr>\n",
" <th>239495</th>\n",
" <td>0</td>\n",
" <td>1.0</td>\n",
" <td>2.90</td>\n",
" <td>263</td>\n",
" <td>2020-02-10 02:44:20</td>\n",
" <td>2020-02-10</td>\n",
" </tr>\n",
" <tr>\n",
" <th>72014</th>\n",
" <td>0</td>\n",
" <td>6.0</td>\n",
" <td>16.05</td>\n",
" <td>233</td>\n",
" <td>2020-02-10 04:12:22</td>\n",
" <td>2020-02-10</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" pickup_weekday passenger_count trip_distance PULocationID \\\n",
"225897 0 1.0 1.20 249 \n",
"108301 0 5.0 19.03 132 \n",
"196729 0 6.0 0.38 68 \n",
"239495 0 1.0 2.90 263 \n",
"72014 0 6.0 16.05 233 \n",
"\n",
" tpep_pickup_datetime pickup_date \n",
"225897 2020-02-10 00:23:21 2020-02-10 \n",
"108301 2020-02-10 01:19:01 2020-02-10 \n",
"196729 2020-02-10 01:29:23 2020-02-10 \n",
"239495 2020-02-10 02:44:20 2020-02-10 \n",
"72014 2020-02-10 04:12:22 2020-02-10 "
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"\n",
"path = f\"https://whylabs-public.s3.us-west-2.amazonaws.com/whylogs_examples/nyc_taxi_rides_feb_2020_changed.parquet\"\n",
"print(f\"Loading data from {path}\")\n",
"rides_df = pd.read_parquet(path)\n",
"\n",
"rides_df.head()"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"rides_df['passenger_count'] = rides_df['passenger_count'].fillna(0).astype('int64')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "S3sEq_qpXQUj"
},
"source": [
"## Additional Transformations"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ismQ0mWDXacC"
},
"source": [
"The real dataset doesn't contain information regarding the taxi driver that conducted the ride. Since our goal is to enrich the dataset with driver features from an external data source, we will create a `driver_id` column. For simplicity, let's consider that this dataset contains ride information of only 2 drivers (IDs `1001` and `1002`)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"id": "wiRJFzh0Xcv5"
},
"outputs": [],
"source": [
"import numpy as np\n",
"rides_df['driver_id'] = np.random.randint(1001, 1003, rides_df.shape[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MdVylBImXh5C"
},
"source": [
"## Features: Load, Fetch and Log"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "J4mc__uVXlkz"
},
"source": [
"We will iterate on `rides_df`, where each row represents a point in time in which we will request a prediction. For each request, we will:\n",
"\n",
"- Materialize latest features into our online feature store\n",
"- Get features from the online feature store\n",
"- Join the features from the online store (driver features) with ride features\n",
"- Log features with whylogs into a profile\n",
"\n",
"We'll consider that the __materialization__ job is run hourly. To simulate that, we will call __materialize__ for the last rounded hour, based on the request's timestamp `tpep_pickup_datetime`.\n",
"\n",
"We will iterate through all the requests on the dataset, generate profiles for daily batches of data, and then write the profiles to disk in a binary file for each of the seven days:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 97.08it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 01:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 02:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 101.23it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 03:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 04:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 117.07it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 04:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 05:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 135.54it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 05:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 06:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 141.93it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 154.83it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 159.83it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 123.47it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 95.67it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.11it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 78.79it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 127.50it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 108.23it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 131.21it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 114.58it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 100.24it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 92.09it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 133.12it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 102.93it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 96.86it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 168.66it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-10 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 88.05it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Starting logger for day 2020-02-11 00:00:00....\n",
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-10 23:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 00:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 158.66it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 157.59it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 04:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 05:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 149.97it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 05:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 06:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0it [00:00, ?it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 148.77it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0it [00:00, ?it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 96.71it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0it [00:00, ?it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 100.11it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0it [00:00, ?it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.29it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0it [00:00, ?it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 108.01it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"0it [00:00, ?it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 94.35it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0it [00:00, ?it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 110.94it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0it [00:00, ?it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 119.28it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0it [00:00, ?it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-11 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 98.93it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Starting logger for day 2020-02-12 00:00:00....\n",
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-11 23:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 00:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"0it [00:00, ?it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 111.18it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 03:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 04:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 142.89it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 83.19it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 103.58it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 139.98it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 105.49it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 124.73it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 128.29it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 105.62it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 103.11it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 123.69it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 84.25it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 127.68it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 111.89it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 108.54it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 92.51it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 101.55it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 93.28it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-12 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-12 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 118.25it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Starting logger for day 2020-02-13 00:00:00....\n",
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 01:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 02:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 122.30it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 03:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 04:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 124.17it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 04:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 05:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 141.48it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 05:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 06:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 120.50it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 125.49it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 90.89it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 133.53it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 96.20it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 81.72it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 88.51it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 112.87it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 126.05it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.44it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 140.34it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 104.42it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 148.64it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 131.96it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 129.22it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 108.97it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 111.21it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-13 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 117.89it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Starting logger for day 2020-02-14 00:00:00....\n",
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-13 23:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 00:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 127.16it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 100.66it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 01:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 02:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 138.51it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 05:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 06:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.10it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 146.52it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 133.95it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 92.94it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 123.47it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 117.00it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 104.38it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 144.85it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 50.76it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 80.35it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.47it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 93.62it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 97.90it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 128.72it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 68.97it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 103.13it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 134.36it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-14 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 100.09it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Starting logger for day 2020-02-15 00:00:00....\n",
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-14 23:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 00:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.28it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 134.45it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 01:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 02:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 121.67it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 02:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 03:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 82.45it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 06:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 07:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 131.26it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 100.35it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 106.04it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 124.26it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 121.92it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 124.89it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 107.71it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.57it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 108.74it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 115.71it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 111.62it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 97.34it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 93.39it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 101.70it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 102.01it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-15 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 93.55it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Starting logger for day 2020-02-16 00:00:00....\n",
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-15 23:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 00:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 115.39it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 00:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 01:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 120.52it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 01:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 02:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 113.76it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 04:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 05:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 125.81it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 07:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 08:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 86.17it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 08:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 09:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 107.42it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 09:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 10:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 123.10it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 10:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 11:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 121.85it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 11:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 12:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 131.44it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 12:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 13:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 141.69it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 13:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 14:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 99.13it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 14:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 15:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 98.62it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 15:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 16:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 149.06it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 16:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 17:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 112.62it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 17:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 18:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 92.72it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 18:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 19:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 102.09it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 19:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 20:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 131.01it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 20:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 21:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 136.77it/s]\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 21:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 22:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 93.03it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Materializing \u001b[1m\u001b[32m1\u001b[0m feature views from \u001b[1m\u001b[32m2020-02-16 22:00:00-03:00\u001b[0m to \u001b[1m\u001b[32m2020-02-16 23:00:00-03:00\u001b[0m into the \u001b[1m\u001b[32msqlite\u001b[0m online store.\n",
"\n",
"\u001b[1m\u001b[32mdriver_hourly_stats\u001b[0m:\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"100%|████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 135.85it/s]\n"
]
}
],
"source": [
"from datetime import datetime, timedelta\n",
"from pprint import pprint\n",
"from feast import FeatureStore\n",
"import os\n",
"import whylogs as why\n",
"\n",
"store = FeatureStore(repo_path=\".\")\n",
"\n",
"prev_time = datetime(2020, 2, 10, 00, 00)\n",
"target_time = datetime(2020, 2, 10, 1, 00)\n",
"store.materialize(start_date=prev_time,end_date=target_time)\n",
"\n",
"# Initializing logger for the first day\n",
"day_to_log = datetime(2020, 2, 10)\n",
"\n",
"profile = None\n",
"for index,row in rides_df.iterrows():\n",
"\n",
" request_timestamp = row['tpep_pickup_datetime']\n",
"\n",
" # If new request is from the next day, close logger, save profile in-memory and start logger for the next day\n",
" if request_timestamp.day > day_to_log.day:\n",
" # let's write our profiles to whylogs_output folder\n",
" why.write(profile,\"whylogs_output\",\"profile_{}_{}_{}.bin\".format(day_to_log.day,day_to_log.month,day_to_log.year))\n",
" day_to_log = request_timestamp.replace(hour=0, minute=0, second=0, microsecond=0)\n",
" print(\"Starting logger for day {}....\".format(day_to_log))\n",
" profile = None\n",
" if request_timestamp>target_time + timedelta(hours=1):\n",
" target_time = datetime(request_timestamp.year,request_timestamp.month,request_timestamp.day,request_timestamp.hour)\n",
" prev_time = target_time - timedelta(hours=1)\n",
" store.materialize(start_date=prev_time,end_date=target_time)\n",
"\n",
" driver_feature_vector = store.get_online_features(\n",
" features=[\n",
" \"driver_hourly_stats:rate_1m\",\n",
" \"driver_hourly_stats:avg_daily_trips\",\n",
" \"driver_hourly_stats:avg_speed\"\n",
" ],\n",
" entity_rows=[{\"driver_id\": row['driver_id']},],\n",
" ).to_dict()\n",
"\n",
" # Get features from both ride and driver\n",
" assembled_feature_vector = {\n",
" \"pickup_weekday\": row[\"pickup_weekday\"],\n",
" \"passenger_count\": row[\"passenger_count\"],\n",
" \"trip_distance\": row[\"trip_distance\"],\n",
" \"PULocationID\": row[\"PULocationID\"],\n",
" \"driver_avg_daily_trips\": driver_feature_vector[\"avg_daily_trips\"][0],\n",
" \"driver_rate_1m\": driver_feature_vector[\"rate_1m\"][0],\n",
" \"driver_avg_speed\": driver_feature_vector[\"avg_speed\"][0],\n",
"\n",
" }\n",
"\n",
" # Now that we have the complete set of features, model prediction could go here.\n",
"\n",
" # The first time data is logged to a profile, we call log(). For subsequent data to be logged in the same profile, let's use track(), until the daily batch is finished.\n",
" if not profile:\n",
" profile = why.log(row=assembled_feature_vector).profile()\n",
" else:\n",
" profile.track(assembled_feature_vector)\n",
" \n",
"why.write(profile,\"whylogs_output\",\"profile_{}_{}_{}.bin\".format(day_to_log.day,day_to_log.month,day_to_log.year))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's confirm that the profiles for each day was indeed written to the disk:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"profile_10_2_2020.bin\n",
"profile_11_2_2020.bin\n",
"profile_12_2_2020.bin\n",
"profile_13_2_2020.bin\n",
"profile_14_2_2020.bin\n",
"profile_15_2_2020.bin\n",
"profile_16_2_2020.bin\n"
]
}
],
"source": [
"%%sh\n",
"ls whylogs_output"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can rehydrate each of those profiles to check some of the metrics provided in the profile. Let's take the first day as our reference profile:"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>cardinality/est</th>\n",
" <th>cardinality/lower_1</th>\n",
" <th>cardinality/upper_1</th>\n",
" <th>counts/n</th>\n",
" <th>counts/null</th>\n",
" <th>distribution/max</th>\n",
" <th>distribution/mean</th>\n",
" <th>distribution/median</th>\n",
" <th>distribution/min</th>\n",
" <th>distribution/n</th>\n",
" <th>...</th>\n",
" <th>distribution/stddev</th>\n",
" <th>frequent_items/frequent_strings</th>\n",
" <th>ints/max</th>\n",
" <th>ints/min</th>\n",
" <th>type</th>\n",
" <th>types/boolean</th>\n",
" <th>types/fractional</th>\n",
" <th>types/integral</th>\n",
" <th>types/object</th>\n",
" <th>types/string</th>\n",
" </tr>\n",
" <tr>\n",
" <th>column</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>PULocationID</th>\n",
" <td>39.000004</td>\n",
" <td>39.0</td>\n",
" <td>39.001951</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>264.000000</td>\n",
" <td>161.663265</td>\n",
" <td>161.00</td>\n",
" <td>41.00</td>\n",
" <td>98</td>\n",
" <td>...</td>\n",
" <td>64.712997</td>\n",
" <td>[FrequentItem(value='161.000000', est=6, upper...</td>\n",
" <td>264.0</td>\n",
" <td>41.0</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>driver_avg_daily_trips</th>\n",
" <td>17.000001</td>\n",
" <td>17.0</td>\n",
" <td>17.000849</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>44.000000</td>\n",
" <td>29.428571</td>\n",
" <td>30.00</td>\n",
" <td>9.00</td>\n",
" <td>98</td>\n",
" <td>...</td>\n",
" <td>7.729979</td>\n",
" <td>[FrequentItem(value='36.000000', est=14, upper...</td>\n",
" <td>44.0</td>\n",
" <td>9.0</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>driver_avg_speed</th>\n",
" <td>35.000003</td>\n",
" <td>35.0</td>\n",
" <td>35.001750</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>31.139999</td>\n",
" <td>21.182551</td>\n",
" <td>20.98</td>\n",
" <td>13.21</td>\n",
" <td>98</td>\n",
" <td>...</td>\n",
" <td>3.748177</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>driver_rate_1m</th>\n",
" <td>4.000000</td>\n",
" <td>4.0</td>\n",
" <td>4.000200</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>4.000000</td>\n",
" <td>2.540816</td>\n",
" <td>3.00</td>\n",
" <td>1.00</td>\n",
" <td>98</td>\n",
" <td>...</td>\n",
" <td>0.801653</td>\n",
" <td>[FrequentItem(value='3.000000', est=45, upper=...</td>\n",
" <td>4.0</td>\n",
" <td>1.0</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>passenger_count</th>\n",
" <td>6.000000</td>\n",
" <td>6.0</td>\n",
" <td>6.000300</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>6.000000</td>\n",
" <td>1.408163</td>\n",
" <td>1.00</td>\n",
" <td>0.00</td>\n",
" <td>98</td>\n",
" <td>...</td>\n",
" <td>1.199972</td>\n",
" <td>[FrequentItem(value='1.000000', est=77, upper=...</td>\n",
" <td>6.0</td>\n",
" <td>0.0</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>pickup_weekday</th>\n",
" <td>1.000000</td>\n",
" <td>1.0</td>\n",
" <td>1.000050</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.00</td>\n",
" <td>0.00</td>\n",
" <td>98</td>\n",
" <td>...</td>\n",
" <td>0.000000</td>\n",
" <td>[FrequentItem(value='0.000000', est=98, upper=...</td>\n",
" <td>0.0</td>\n",
" <td>0.0</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>trip_distance</th>\n",
" <td>83.000017</td>\n",
" <td>83.0</td>\n",
" <td>83.004161</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>20.220000</td>\n",
" <td>2.791531</td>\n",
" <td>1.62</td>\n",
" <td>0.24</td>\n",
" <td>98</td>\n",
" <td>...</td>\n",
" <td>3.606351</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>SummaryType.COLUMN</td>\n",
" <td>0</td>\n",
" <td>98</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>7 rows × 28 columns</p>\n",
"</div>"
],
"text/plain": [
" cardinality/est cardinality/lower_1 \\\n",
"column \n",
"PULocationID 39.000004 39.0 \n",
"driver_avg_daily_trips 17.000001 17.0 \n",
"driver_avg_speed 35.000003 35.0 \n",
"driver_rate_1m 4.000000 4.0 \n",
"passenger_count 6.000000 6.0 \n",
"pickup_weekday 1.000000 1.0 \n",
"trip_distance 83.000017 83.0 \n",
"\n",
" cardinality/upper_1 counts/n counts/null \\\n",
"column \n",
"PULocationID 39.001951 98 0 \n",
"driver_avg_daily_trips 17.000849 98 0 \n",
"driver_avg_speed 35.001750 98 0 \n",
"driver_rate_1m 4.000200 98 0 \n",
"passenger_count 6.000300 98 0 \n",
"pickup_weekday 1.000050 98 0 \n",
"trip_distance 83.004161 98 0 \n",
"\n",
" distribution/max distribution/mean \\\n",
"column \n",
"PULocationID 264.000000 161.663265 \n",
"driver_avg_daily_trips 44.000000 29.428571 \n",
"driver_avg_speed 31.139999 21.182551 \n",
"driver_rate_1m 4.000000 2.540816 \n",
"passenger_count 6.000000 1.408163 \n",
"pickup_weekday 0.000000 0.000000 \n",
"trip_distance 20.220000 2.791531 \n",
"\n",
" distribution/median distribution/min distribution/n \\\n",
"column \n",
"PULocationID 161.00 41.00 98 \n",
"driver_avg_daily_trips 30.00 9.00 98 \n",
"driver_avg_speed 20.98 13.21 98 \n",
"driver_rate_1m 3.00 1.00 98 \n",
"passenger_count 1.00 0.00 98 \n",
"pickup_weekday 0.00 0.00 98 \n",
"trip_distance 1.62 0.24 98 \n",
"\n",
" ... distribution/stddev \\\n",
"column ... \n",
"PULocationID ... 64.712997 \n",
"driver_avg_daily_trips ... 7.729979 \n",
"driver_avg_speed ... 3.748177 \n",
"driver_rate_1m ... 0.801653 \n",
"passenger_count ... 1.199972 \n",
"pickup_weekday ... 0.000000 \n",
"trip_distance ... 3.606351 \n",
"\n",
" frequent_items/frequent_strings \\\n",
"column \n",
"PULocationID [FrequentItem(value='161.000000', est=6, upper... \n",
"driver_avg_daily_trips [FrequentItem(value='36.000000', est=14, upper... \n",
"driver_avg_speed NaN \n",
"driver_rate_1m [FrequentItem(value='3.000000', est=45, upper=... \n",
"passenger_count [FrequentItem(value='1.000000', est=77, upper=... \n",
"pickup_weekday [FrequentItem(value='0.000000', est=98, upper=... \n",
"trip_distance NaN \n",
"\n",
" ints/max ints/min type types/boolean \\\n",
"column \n",
"PULocationID 264.0 41.0 SummaryType.COLUMN 0 \n",
"driver_avg_daily_trips 44.0 9.0 SummaryType.COLUMN 0 \n",
"driver_avg_speed NaN NaN SummaryType.COLUMN 0 \n",
"driver_rate_1m 4.0 1.0 SummaryType.COLUMN 0 \n",
"passenger_count 6.0 0.0 SummaryType.COLUMN 0 \n",
"pickup_weekday 0.0 0.0 SummaryType.COLUMN 0 \n",
"trip_distance NaN NaN SummaryType.COLUMN 0 \n",
"\n",
" types/fractional types/integral types/object \\\n",
"column \n",
"PULocationID 0 98 0 \n",
"driver_avg_daily_trips 0 98 0 \n",
"driver_avg_speed 98 0 0 \n",
"driver_rate_1m 0 98 0 \n",
"passenger_count 0 98 0 \n",
"pickup_weekday 0 98 0 \n",
"trip_distance 98 0 0 \n",
"\n",
" types/string \n",
"column \n",
"PULocationID 0 \n",
"driver_avg_daily_trips 0 \n",
"driver_avg_speed 0 \n",
"driver_rate_1m 0 \n",
"passenger_count 0 \n",
"pickup_weekday 0 \n",
"trip_distance 0 \n",
"\n",
"[7 rows x 28 columns]"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"reference_profile = why.read(os.path.join(\"whylogs_output\",\"profile_10_2_2020.bin\"))\n",
"# we generate a profile view, and then call to_pandas() to have a dataframe with the metrics to be inspected\n",
"reference_metrics = reference_profile.view().to_pandas()\n",
"reference_metrics"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you want to know more about inspecting profiles and metrics contained in them, check the example on [Inspecting Profiles](https://whylogs.readthedocs.io/en/stable/examples/basic/Inspecting_Profiles.html) !"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Injecting data issues and comparing profiles"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, let's add some data error issues into the dataset and see how we could visually inspect this with some of whylog's functionalites. Some of the changes applied are shown as following:\n",
"\n",
"- Feb 10: No changes\n",
"\n",
"- Feb 11: (Data update error) New driver features are available only in 2 hour cycles. Simulating a scenario in which the sampling frequency is affected due to changes upstream.\n",
"- Feb 16: (Feature drift) Based on the considerations made on section Changes in Data, we will: a) Reduce the number of passengers (passenger_count) and b) increase the standard deviation of rate_1m's distribution. For more information of how that was done, please see Appendix - Changing the Dataset."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We already have our reference profile, so let's load from disk two other profiles that contain the data update and feature drift issues, respectively."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"target_profile_1 = why.read(os.path.join(\"whylogs_output\",\"profile_11_2_2020.bin\"))\n",
"target_profile_2 = why.read(os.path.join(\"whylogs_output\",\"profile_16_2_2020.bin\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Data update"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The data update issue is a subtle one, since we still have data available with the expected shape and values. The only difference is that the values are being updated less often. Ideally, this could be checked elsewhere in our pipeline, but with information available in our assembled feature vector, we could get signals of this issues indirectly by inspecting the cardinality of the features collected from the driver source.\n",
"\n",
"Let's check the cardinality of the average speed for our reference profile, which is a float variable:\n"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cardinality for driver average speed for Reference dataset: 35.000002955397264\n"
]
}
],
"source": [
"card = reference_metrics.loc['driver_avg_speed']['cardinality/est']\n",
"print(\"Cardinality for driver average speed for Reference dataset:\",card)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For the same frequency update, we can expect cardinality estimates around the value seen in our baseline.\n",
"\n",
"Let's now compare the cardinality estimations in the other two profiles:"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cardinality for driver average speed for profile #1:\n",
"24.00000137090692\n",
"Cardinality for driver average speed for profile #2:\n",
"35.000002955397264\n"
]
}
],
"source": [
"profile_1_metrics = target_profile_1.view().to_pandas() \n",
"profile_2_metrics = target_profile_2.view().to_pandas() \n",
"\n",
"print(\"Cardinality for driver average speed for profile #1:\")\n",
"print(profile_1_metrics.loc['driver_avg_speed']['cardinality/est'])\n",
"print(\"Cardinality for driver average speed for profile #2:\")\n",
"print(profile_2_metrics.loc['driver_avg_speed']['cardinality/est'])"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We can see there's a significant difference for the profile that is updated less frequently.\n",
"\n",
"You could automate this type of assertion by using Constraints in order to do data validation in your data. If you want to know more, please see the example on [Building Metric Constraints](https://whylogs.readthedocs.io/en/stable/examples/advanced/Metric_Constraints.html)!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Feature Drift"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For February 16, we have a change in the average daily trips and also in the driver's monthly rating.\n",
"\n",
"We can compare both profiles in order to detect data drifts and generate a report for every feature in the profiles by using the `NotebookProfileVisualizer`"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"from whylogs.viz import NotebookProfileVisualizer\n",
"\n",
"visualization = NotebookProfileVisualizer()\n",
"visualization.set_profiles(target_profile_view=target_profile_2.view(), reference_profile_view=reference_profile.view())"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div></div><iframe srcdoc=\"<!DOCTYPE html>\n",
"<html lang="en">\n",
" <head>\n",
" <meta charset="UTF-8" />\n",
" <meta http-equiv="X-UA-Compatible" content="IE=edge" />\n",
" <meta name="viewport" content="width=device-width, initial-scale=1.0" />\n",
" <meta name="description" content="" />\n",
" <meta name="author" content="" />\n",
"\n",
" <title>Profile Visualizer | whylogs</title>\n",
"\n",
" <link rel="icon" href="images/whylabs-favicon.png" type="image/png" sizes="16x16" />\n",
" <link rel="preconnect" href="https://fonts.googleapis.com" />\n",
" <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />\n",
" <link href="https://fonts.googleapis.com/css2?family=Asap:wght@400;500;600;700&display=swap" rel="stylesheet" />\n",
" <link rel="preconnect" href="https://fonts.gstatic.com" />\n",
" <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.0/dist/css/bootstrap.min.css" />\n",
"\n",
" <script\n",
" src="https://cdnjs.cloudflare.com/ajax/libs/handlebars.js/4.7.7/handlebars.min.js"\n",
" integrity="sha512-RNLkV3d+aLtfcpEyFG8jRbnWHxUqVZozacROI4J2F1sTaDqo1dPQYs01OMi1t1w9Y2FdbSCDSQ2ZVdAC8bzgAg=="\n",
" crossorigin="anonymous"\n",
" referrerpolicy="no-referrer"\n",
" ></script>\n",
"\n",
" <style type="text/css">\n",
" :root {\n",
" /* CONSTANTS */\n",
" --SIDE-PANEL-WIDTH: 320px;\n",
" --PROPERTY-PANEL-WIDTH: 420px;\n",
"\n",
" /* COLOR VARIABLES */\n",
" /** Standard colors */\n",
" --red: #d11010;\n",
" --orange: #f07028;\n",
" --yellow: #faaf40;\n",
" --olive: #b5cc18;\n",
" --green: #1dbb42;\n",
" --teal: #00b5ad;\n",
" --blue: #2683c9;\n",
" --violet: #6435c9;\n",
" --purple: #a333c8;\n",
" --pink: #ed45a4;\n",
" --brown: #ac724d;\n",
" --grey: #778183;\n",
" --black: #1b1c1d;\n",
" --white: #ffffff;\n",
"\n",
" /** Branded colors */\n",
" --brandPrimary900: #0e7384;\n",
" --brandPrimary800: #228798;\n",
" --brandPrimary700: #369bac;\n",
" --brandPrimary600: #4aafc0;\n",
" --brandPrimary500: #5ec3d4;\n",
" --brandPrimary400: #72d7e8;\n",
" --brandPrimary300: #86ebfc;\n",
" --brandPrimary200: #a6f2ff;\n",
" --brandPrimary100: #cdf8ff;\n",
" --brandSecondary900: #4f595b;\n",
" --brandSecondary800: #636d6f;\n",
" --brandSecondary700: #778183;\n",
" --brandSecondary600: #8b9597;\n",
" --brandSecondary500: #9fa9ab;\n",
" --brandSecondary400: #b3bdbf;\n",
" --brandSecondary300: #c7d1d3;\n",
" --brandSecondary200: #dbe5e7;\n",
" --brandSecondary100: #ebf2f3;\n",
" --secondaryLight1000: #313b3d;\n",
" --brandRed4: #b30000;\n",
" --brandRed3: #d72424;\n",
" --brandRed2: #eb5656;\n",
" --brandRed1: #ff8282;\n",
" --night1: #021826;\n",
" /** Purpose colors */\n",
" --textColor: #4f595b;\n",
" --linkColor: #369bac;\n",
" --infoColor: #2683c9;\n",
" --warningColor: #faaf40;\n",
" --tealBackground: #eaf2f3;\n",
" --pageBackground: #e5e5e5;\n",
" --contrastTableRow: #fafafa;\n",
" --whiteBackground: #ffffff;\n",
" --primaryBackground: #e5e5e5;\n",
" }\n",
"\n",
" /* RESET STYLE */\n",
" *,\n",
" *::after,\n",
" *::before {\n",
" margin: 0;\n",
" padding: 0;\n",
" box-sizing: border-box;\n",
" font-family: "Asap", Arial, Helvetica, sans-serif;\n",
" }\n",
"\n",
" /*\n",
" * Main content\n",
" */\n",
" .main {\n",
" position: relative;\n",
" background: #FFFFFF;\n",
" border: 1px solid #DBE5E7;\n",
" box-sizing: border-box;\n",
" border-radius: 4px;\n",
" }\n",
" .main .page-header {\n",
" margin-top: 0;\n",
" }\n",
"\n",
" /* CSS DIV TABLE BASIC STYLE */\n",
"\n",
" .wl-table-row .wl-table-head:first-child {\n",
" min-width: 360px;\n",
" z-index: 2;\n",
" }\n",
"\n",
" .wl-table-wrap {\n",
" position: relative;\n",
" }\n",
"\n",
" .wl-table {\n",
" display: table;\n",
" width: 100%;\n",
" }\n",
"\n",
" .row>* {\n",
" padding-right: 0 !important;\n",
" padding-left: 0 !important;\n",
" }\n",
"\n",
" .wl-table-row {\n",
" display: table-row;\n",
" }\n",
"\n",
" .wl-table-row:hover,\n",
" .wl-table-row:hover .wl-table-cell:first-child {\n",
" background-color: var(--brandSecondary100);\n",
" }\n",
" .wl-table-row--clickable:hover .wl-table-cell__title-button {\n",
" visibility: visible;\n",
" }\n",
"\n",
" .wl-table-heading {\n",
" position: sticky;\n",
" top: 0;\n",
" z-index: 900;\n",
" display: table-header-group;\n",
" font-weight: 700;\n",
" }\n",
"\n",
" .wl-table-cell,\n",
" .wl-table-head {\n",
" border-bottom: 1px solid var(--brandSecondary200);\n",
" display: table-cell;\n",
" padding: 12px 18px;\n",
" }\n",
"\n",
" .wl-table-cell {\n",
" font-size: 14px;\n",
" }\n",
"\n",
" .wl-table-cell--top-spacing {\n",
" padding-top: 35px; /* cell-top-padding + cell-title-height */\n",
" }\n",
"\n",
" .wl-table-head-wraper {\n",
" background-color: var(--white);\n",
" }\n",
"\n",
" .wl-table-head {\n",
" position: relative;\n",
" left: 0;\n",
" min-width: 100px;\n",
" border-bottom: 2px solid var(--brandSecondary100);\n",
" font-size: 12px;\n",
" line-height: 1.67;\n",
" white-space: nowrap;\n",
" }\n",
"\n",
" .wl-sub-table-head {\n",
" position: relative;\n",
" }\n",
"\n",
" .wl-table-body {\n",
" display: table-row-group;\n",
" }\n",
"\n",
" .wl-table-row .wl-table-cell:first-child{\n",
" position: relative;\n",
" left: 0;\n",
" background-color: var(--white);\n",
" border-right-width: 2px;\n",
" }\n",
"\n",
" /* Table custom style */\n",
"\n",
" .wl-table-cell__title-wrap {\n",
" display: flex;\n",
" justify-content: space-between;\n",
" }\n",
"\n",
" .wl-table-cell__title {\n",
" height: 25px;\n",
" margin: 0;\n",
" font-size: 14px;\n",
" font-weight: 700;\n",
" overflow: hidden;\n",
" white-space: nowrap;\n",
" text-overflow: ellipsis;\n",
" }\n",
"\n",
" .wl-table-cell__bedge-wrap {\n",
" padding: 2px 0;\n",
" white-space: nowrap;\n",
" overflow: hidden;\n",
" font-style: italic;\n",
" text-align: center;\n",
" color: var(--brandSecondary400);\n",
" }\n",
" .wl-table-cell__bedge {\n",
" height: 24px;\n",
" margin: 1px;\n",
" padding: 2px 8px;\n",
" border: 1px solid var(--brandSecondary400);\n",
" font-style: normal;\n",
" color: var(--brandSecondary900);\n",
" border-radius: 20px;\n",
" white-space: nowrap;\n",
" }\n",
"\n",
" /* Property side panel */\n",
"\n",
" .wl-compare-profile {\n",
" position: relative;\n",
" left: 0;\n",
" padding: 18px;\n",
" background: var(--white);;\n",
" border-bottom: 1px solid var(--brandSecondary200);\n",
" }\n",
"\n",
" /* Screen on smaller screens */\n",
" .no-responsive {\n",
" display: none;\n",
" position: fixed;\n",
" top: 0;\n",
" left: 0;\n",
" z-index: 1031;\n",
" width: 100vw;\n",
" height: 100vh;\n",
" background-color: var(--tealBackground);\n",
" display: flex;\n",
" align-items: center;\n",
" justify-content: center;\n",
" }\n",
"\n",
" .no-responsive__content {\n",
" max-width: 600px;\n",
" width: 100%;\n",
" padding: 0 24px;\n",
" }\n",
"\n",
" .no-responsive__title {\n",
" font-size: 96px;\n",
" font-weight: 300;\n",
" color: var(--brandSecondary900);\n",
" line-height: 1.167;\n",
" }\n",
"\n",
" .no-responsive__text {\n",
" margin: 0;\n",
" font-size: 16px;\n",
" font-weight: 400;\n",
" color: var(--brandSecondary900);\n",
" line-height: 1.5;\n",
" }\n",
"\n",
" .space-between {\n",
" display: flex;\n",
" justify-content: space-between;\n",
" }\n",
"\n",
" .align-items {\n",
" display: flex;\n",
" align-items: center;\n",
" }\n",
"\n",
" .display-flex{\n",
" display: flex;\n",
" }\n",
"\n",
" .table-border-none {\n",
" padding: 0;\n",
" border: none;\n",
" }\n",
"\n",
" .flex-direction-colum {\n",
" display: flex;\n",
" flex-direction: column;\n",
" }\n",
"\n",
" .align-items {\n",
" align-items: center;\n",
" }\n",
"\n",
" .search-input{\n",
" padding-top: 0 !important;\n",
" padding-bottom: 0 !important;\n",
" }\n",
"\n",
" .search-input input{\n",
" border: none;\n",
" background: none;\n",
" outline: none;\n",
" height: 40px;\n",
" width: 100%;\n",
" font-size: 14px;\n",
" }\n",
"\n",
" .search-input img{\n",
" height: 19px;\n",
" pointer-events: none;\n",
" }\n",
"\n",
" input::placeholder {\n",
" color: var(--secondaryLight1000);\n",
" }\n",
"\n",
" .text-align-center {\n",
" text-align: center;\n",
" }\n",
"\n",
" .text-align-end {\n",
" text-align: end;\n",
" }\n",
"\n",
" .drift-detection {\n",
" justify-content: space-between;\n",
" align-items: center;\n",
" }\n",
"\n",
" .drift-detection-info-circle {\n",
" width: 15px;\n",
" height: 15px;\n",
" border-radius: 50px;\n",
" display: inline-block;\n",
" margin-right: 8px;\n",
" }\n",
"\n",
" .severe-drift-circle-color {\n",
" background: #D40D00;\n",
" }\n",
"\n",
" .moderate-drift-circle-color {\n",
" background: #F5843C;\n",
" }\n",
"\n",
" .mild-drift-circle-color {\n",
" background: #F2C142;\n",
" }\n",
"\n",
" .minimal-drift-circle-color {\n",
" background: #ABCA52;\n",
" }\n",
"\n",
" .drift-detection-info-drifts-item {\n",
" padding-right: 20px;\n",
" }\n",
"\n",
" .drift-detection-info-title {\n",
" font-family: Arial;\n",
" font-weight: bold;\n",
" font-size: 16px;\n",
" line-height: 130%;\n",
" color: #313B3D;\n",
" }\n",
"\n",
" .drift-detection-info-drifts-item-count {\n",
" font-family: Arial;\n",
" font-weight: bold;\n",
" font-size: 14px;\n",
" line-height: 16px;\n",
" color: #000000;\n",
" padding-right: 8px;\n",
" }\n",
"\n",
" .drift-detection-info-drifts-item-name {\n",
" font-family: Arial;\n",
" font-style: normal;\n",
" font-weight: normal;\n",
" font-size: 12px;\n",
" line-height: 14px;\n",
" color: #000000;\n",
" }\n",
"\n",
" .drift-detection-info-drifts-item-range {\n",
" font-family: Arial;\n",
" font-style: normal;\n",
" font-weight: normal;\n",
" font-size: 11px;\n",
" line-height: 13px;\n",
" color: #6C757D;\n",
" }\n",
"\n",
" .drift-detection-search-input {\n",
" display: flex;\n",
" align-items: center;\n",
" background: rgba(255, 255, 255, 0.7);\n",
" border: 1px solid #DBE5E7;\n",
" box-sizing: border-box;\n",
" border-radius: 4px;\n",
" width: 170px;\n",
" padding-left: 10px;\n",
" }\n",
"\n",
" .drift-detection-search-input img{\n",
" margin-right: 5px;\n",
" }\n",
"\n",
" .drift-detection-search-input input::placeholder {\n",
" font-family: Arial;\n",
" font-weight: normal;\n",
" font-size: 13px;\n",
" line-height: 16px;\n",
" color: #313B3D;\n",
" }\n",
"\n",
" .close-filter-button {\n",
" display: flex;\n",
" justify-content: center;\n",
" align-items: center;\n",
" background: rgba(255, 255, 255, 0.7);\n",
" border: 1px solid #369BAC;\n",
" box-sizing: border-box;\n",
" border-radius: 4px;\n",
" width: 40px;\n",
" height: 40px;\n",
" cursor: pointer;\n",
" margin-left: 10px;\n",
" }\n",
"\n",
" .filter-options-title {\n",
" width: 240px;\n",
" }\n",
"\n",
" .filter-options-title p {\n",
" margin: 0;\n",
" }\n",
"\n",
" .dropdown-container {\n",
" position: absolute;\n",
" right: 18px;\n",
" z-index: 999;\n",
" background: #FFFFFF;\n",
" border: 1px solid #DBE5E7;\n",
" box-sizing: border-box;\n",
" box-shadow: 0px 2px 8px rgba(0, 0, 0, 0.05);\n",
" border-radius: 4px;\n",
" padding: 10px !important;\n",
" border: none !important;\n",
" }\n",
"\n",
" .form-check-input:checked {\n",
" background-color: #0E7384;\n",
" border-color: #0E7384;\n",
" }\n",
"\n",
" .form-check-input[type=checkbox] {\n",
" border-radius: 2px;\n",
" }\n",
"\n",
" .justify-content-center {\n",
" justify-content: center;\n",
" }\n",
"\n",
" .wl-table-cell__graph-wrap {\n",
" width: 0;\n",
" }\n",
"\n",
" .svg-container {\n",
" display: inline-block;\n",
" position: relative;\n",
" width: 85%;\n",
" padding-bottom: 17%;\n",
" vertical-align: top;\n",
" overflow: hidden;\n",
" }\n",
"\n",
" .svg-content-responsive {\n",
" display: inline-block;\n",
" position: absolute;\n",
" left: 0;\n",
" }\n",
"\n",
" .reference-table-head {\n",
" min-width: 250px;\n",
" }\n",
"\n",
" .wl__dropdown_arrow-icon {\n",
" position: relative;\n",
" }\n",
"\n",
" .notif-circle-container{\n",
" position: absolute;\n",
" top: -4px;\n",
" right: -4px;\n",
" padding: 5.3px;\n",
" border-radius: 50%;\n",
" background-color: white;\n",
" cursor: pointer;\n",
" }\n",
"\n",
" .notif-circle {\n",
" position: absolute;\n",
" top: 2px;\n",
" right: 2px;\n",
" padding: 3.3px;\n",
" border-radius: 50%;\n",
" background-color: #F2994A;\n",
" }\n",
"\n",
"\n",
" .header-title {\n",
" font-size: 26px;\n",
" font-weight: 700;\n",
" color: #444444;\n",
" }\n",
"\n",
" .statistic-number-title {\n",
" font-family: Arial;\n",
" font-weight: normal;\n",
" font-size: 14px;\n",
" line-height: 20px;\n",
" color: #6C757D;\n",
" }\n",
"\n",
" .statistic-number {\n",
" font-family: Arial;\n",
" font-weight: bold;\n",
" font-size: 20px;\n",
" line-height: 140%;\n",
" display: flex;\n",
" align-items: center;\n",
" color: #0E7384;\n",
" }\n",
"\n",
" .statistic-measurement {\n",
" font-size: 15px !important;\n",
" margin-left: 3px;\n",
" }\n",
"\n",
" .statistic-measurement-percent {\n",
" font-size: 15px !important;\n",
" }\n",
"\n",
" .question-mark {\n",
" font-size: 10px;\n",
" font-weight: 900;\n",
" color: #0E7384;\n",
" border-radius: 50px;\n",
" border: 2px solid #0E7384;\n",
" padding: 0px 4px;\n",
" transition: 0.5s;\n",
" cursor: pointer;\n",
" }\n",
"\n",
" .question-mark:hover {\n",
" color: white;\n",
" background: #0E7384;\n",
" border: 2px solid none;\n",
" transition: 0.5s;\n",
" }\n",
"\n",
" .tooltip-full-number {\n",
" position: relative;\n",
" display: inline-block;\n",
" }\n",
"\n",
" .tooltip-full-number .tooltiptext {\n",
" visibility: hidden;\n",
" background: black;\n",
" color: white;\n",
" border: 1px solid black;\n",
" text-align: start;\n",
" padding: 3px;\n",
" position: absolute;\n",
" z-index: 1;\n",
" top: 0;\n",
" left: 100%;\n",
" margin-left: 5px;\n",
" opacity: 0;\n",
" transition: opacity 0.5s;\n",
" font-size: 13px;\n",
" font-weight: normal;\n",
" line-height: 100%;\n",
" }\n",
"\n",
" .tooltip-full-number:hover .tooltiptext {\n",
" visibility: visible;\n",
" opacity: 1;\n",
" }\n",
"\n",
" .display-flex {\n",
" display: flex;\n",
" }\n",
"\n",
" .flex-direction-column {\n",
" flex-direction: column;\n",
" }\n",
"\n",
" .justify-content-space-between {\n",
" justify-content: space-between;\n",
" }\n",
"\n",
" .justify-content-center {\n",
" justify-content: center;\n",
" }\n",
"\n",
" .align-items-center {\n",
" align-items: center;\n",
" }\n",
"\n",
" .padding-right-30 {\n",
" padding-right: 30px;\n",
" }\n",
"\n",
" .padding-5 {\n",
" padding: 5px;\n",
" }\n",
"\n",
" .text-color {\n",
" color: var(--secondaryLight1000);\n",
" }\n",
"\n",
" .error-message {\n",
" display: flex;\n",
" justify-content: center;\n",
" align-items: center;\n",
" color: rgb(255, 114, 71);\n",
" font-size: 30px;\n",
" font-weight: 900;\n",
" }\n",
"\n",
" @media screen and (min-width: 500px) {\n",
" .desktop-content {\n",
" display: block;\n",
" }\n",
" .no-responsive {\n",
" display: none;\n",
" }\n",
" }\n",
" </style>\n",
" </head>\n",
"\n",
" <body id="generated-html"></body>\n",
"\n",
" <script id="entry-template" type="text/x-handlebars-template">\n",
" \n",
" <div class="desktop-content">\n",
" <div class="container-fluid">\n",
" <div class="feature-summary-statistics-wrap">\n",
" <div class="feature-summary-statistics">\n",
" <div class="mb-4">\n",
" <strong class="header-title">Profile Summary</strong>\n",
" </div>\n",
" <div class="display-flex statistics">\n",
" <div class="padding-right-30">\n",
" <div class="statistic-number-title">Observations</div>\n",
" <div class="statistic-number">{{{observations this}}}</div>\n",
" </div>\n",
" <div class="padding-right-30">\n",
" <div class="statistic-number-title">Missing Cells</div>\n",
" <div class="statistic-number">\n",
" {{{missingCells this}}}\n",
" <div>{{{missingCellsPercentage this}}}</div>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" <div class="row">\n",
" <div class="main">\n",
" <div class="wl-compare-profile" id="compare-profile">\n",
" <div class="drift-detection-wrap">\n",
" <div class="drift-detection display-flex">\n",
" <div class="drift-detection-info flex-direction-colum">\n",
" <div class="drift-detection-info-title-wrap display-flex">\n",
" <p class="drift-detection-info-title">\n",
" Drift detected in\n",
" <span class="drift-count"></span>\n",
" of\n",
" <span class="all-features"></span>\n",
" features\n",
" </p>\n",
" </div>\n",
" <div class="drift-detection-info-drifts display-flex" id="drift-detection-info-drifts">\n",
" </div>\n",
" </div>\n",
" <div class="drift-detection-search-input-wrap display-flex">\n",
" <div class="drift-detection-search-input search-input">\n",
" <input type="text" id="wl__feature-search" placeholder="Quick search..."/>\n",
" <img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAGdSURBVHgBpZK7TkJBEIZnZoVocdTYyQNALxpNKPQBMEaNJsbCRMKhl8ISWwt6AaksCF5iTHgAGhJD4AHkAaAzGiwUsjvOQnO4SYh/cXbPzO43OxcEj9Zy92EiFSXNIfvPyE1kKFfdoxJMENpP6DrvLC0vJoEwCgwto7DWcxoIIHBYbA3NmKwnDltjAeuZhyul1DaTTlfPB6Nt5Z53DOgky4P875+nlctY2+unjZviLklkJhi5bPUa3y/7qJuQUM7PinMy7CdQc1Gh16vnBxPzrMROmlKQEgKNASAHLQCmSIGpS75O+O5pdQAgVXaIqTkNwDDXHmcnW3VmHZoGMLoTsOt88+NrAMCIZdu+iLTyTwKRa1Md6YKfOgXbzO7K8sWku5u5RxcRV5EpPezrzcHGbXEXWaUkgkweZ/UC9YrK3zqggFw5FBZfm8EUavHj7AjAKpIvBDrGn+pNnlcyhYgqbcC41idr1gvB4SdZkDbzQa21gwv0Vj07aPTtL07XdDOyDXohCDNoHIRmAVRie20f+RKybRDQDvxHkXy/7b/DrayncLbMwQAAAABJRU5ErkJggg=="/>\n",
" </div>\n",
" <div class="wl__dropdown_arrow-icon">\n",
" <div onclick="openFilter()" class="close-filter-button">\n",
" <div class="display-flex close-filter-icon d-none">\n",
" <img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAALCAYAAACprHcmAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAADFSURBVHgBfY+xDcIwEEXvnLQBZQkYAEhDwwKpEEK0CCZgAEjJCEmgjYSAygxAHTZgFRSOsyUjY5mcZFnn/+78PwBXf3+MoKWUPuYjVBPFnTwpr9t/oNJfcTfXsAhRAlDqDhhQIPYgpAqNMDqcUqSAYZT1epr9gAHt6uXshvYme4DYHQJNDKh0dD0m5WXB10Y3Fqjtuh7fROn3oREDWxfeMLyRsMnc0OgDzdduaA0Pi3Plgr7Q2kaAePeBqh6rueSNBVt6fgCwBV1JLF3rlAAAAABJRU5ErkJggg=="/>\n",
" </div>\n",
" <div class="display-flex filter-icon">\n",
" <img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAPCAYAAADtc08vAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAACSSURBVHgBrZLBCYAwDEWTUjw7igcdwI1cxkNXUJzBEVzFAbSVKoKmaVrEB6GHJv/w+AACtRk7P9IONv1QOYUl96k0zv61m2tjARoLtSDI3EFsgIJ4uoXrMLazO72CRG2mzg/8BSdVlEjhpGZJjAWdAZJECpWalEhJSs1pHuUlMad5FFai1Lwg4Ckx1TxKIPFL8w55mEWd8VjPGAAAAABJRU5ErkJggg=="/>\n",
" </div>\n",
" </div>\n",
" <span class="notif-circle-container">\n",
" <span class="notif-circle"></span>\n",
" </span>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" <div class="dropdown-container flex-direction-colum mb-2 d-none" id="dropdown-container">\n",
" <div class="filter-options">\n",
" <div class="filter-options-title space-between dropdown">\n",
" <p>Filter by type</p>\n",
" </div>\n",
" <div class="form-check mb-1 mt-2">\n",
" <input\n",
" class="form-check-input wl__feature-filter-input"\n",
" type="checkbox"\n",
" name="checkbox"\n",
" value="Discrete"\n",
" id="inferredDiscrete"\n",
" onclick="changeDiscreteValue()"\n",
" checked\n",
" />\n",
" <label class="form-check-label" for="inferredDiscrete">\n",
" Inferred discrete (<span class="wl__feature-count--discrete">{{getDiscreteTypeCount}}</span>)\n",
" </label>\n",
" </div>\n",
" <div class="form-check mb-1">\n",
" <input\n",
" class="form-check-input wl__feature-filter-input"\n",
" type="checkbox"\n",
" name="checkbox"\n",
" value="Non-discrete"\n",
" id="inferredNonDiscrete"\n",
" onclick="changeNonDiscreteValue()"\n",
" checked\n",
" />\n",
" <label class="form-check-label" for="inferredNonDiscrete">\n",
" Inferred non-discrete (<span\n",
" class="wl__feature-count--non-discrete"\n",
" >{{getNonDiscreteTypeCount}}</span>)\n",
" </label>\n",
" </div>\n",
" <div class="form-check mb-1">\n",
" <input\n",
" class="form-check-input wl__feature-filter-input"\n",
" type="checkbox"\n",
" name="checkbox"\n",
" value="Unknown"\n",
" id="inferredUnknown"\n",
" onclick="changeUnknwonValue()"\n",
" checked\n",
" />\n",
" <label class="form-check-label" for="inferredUnknown">\n",
" Unknown (<span class="wl__feature-count--unknown">{{getUnknownTypeCount}}</span>)\n",
" </label>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" <div class="wl-table-wrap" id="table-content">\n",
" <div class="wl-table">\n",
" <div class="wl-table-heading">\n",
" <div class="wl-table-row wl-table-row--bottom-shadow">\n",
" <div class="wl-table-head wl-table-head-wraper">\n",
" <div class="wl-table-head table-border-none graph-table-head">Target</div>\n",
" <div class="wl-table-head table-border-none reference-table-head">Reference</div>\n",
" </div>\n",
" <div class="wl-table-head wl-table-head-wraper text-align-end" style="cursor: pointer;" id="diff-from-ref">\n",
" <p id="pvalue" > p-value </p>\n",
" <p id="pvalue-asc" style="display: none;"> p-value &#9650 </p>\n",
" <p id="pvalue-desc" style="display: none;"> p-value &#9660 </p>\n",
"\n",
" <div class="tooltip-full-number">\n",
" <span class="question-mark">?</span>\n",
" <span class="tooltiptext">\n",
" <div class="mb-1">p-values are calculated with K-S test for numerical features and Chi-squared for categorical features.</div>\n",
" </span>\n",
" </div>\n",
" </div>\n",
" <div class="wl-table-head wl-table-head-wraper text-align-end">Total count</div>\n",
" <div class="wl-table-head wl-table-head-wraper text-align-end">Mean</div>\n",
" </div>\n",
" </div>\n",
" <ul class="wl-table-body wl__table-body" id="table-body">\n",
"{{#each this.columns}} <li\n",
" {{#if this.numberSummary}} class="wl-table-row wl-table-row--clickable" {{else}} class="wl-table-row" {{/if}}\n",
" data-feature-name={{@key}}\n",
" data-inferred-type={{inferredType this}}\n",
" data-scroll-to-feature-name={{@key}}\n",
" data-p-value={{getpvalue this}}\n",
" >\n",
" <div class="wl-table-cell wl-table-cell__graph-wrap">\n",
" <div class="wl-table-cell__title-wrap">\n",
" <h4 class="wl-table-cell__title">{{@key}}</h4>\n",
" <div></div>\n",
" </div>\n",
" <div class="display-flex">\n",
" {{{getGraphHtml this}}}\n",
" {{{getReferenceGraphHtml this}}}\n",
" </div>\n",
" </div>\n",
" <div\n",
" class="diff-from-ref-table-cell wl-table-cell wl-table-cell--top-spacing align-middle"\n",
" style="max-width: 270px; padding-right: 18px"\n",
" ><div class="wl-table-cell__bedge-wrap text-align-end">\n",
" {{{getDiffFromRef this}}}\n",
" </div></div><div\n",
" class="wl-table-cell wl-table-cell--top-spacing align-middle"\n",
" ><div class="text-align-end">{{totalCount this}}</div></div><div\n",
" class="wl-table-cell wl-table-cell--top-spacing align-middle"\n",
" ><div class="text-align-end">{{mean this}}</div></div>\n",
" </li>\n",
"{{/each}} </ul>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" <div class="no-responsive">\n",
" <div class="no-responsive__content">\n",
" <h1 class="no-responsive__title">Hold on! :)</h1>\n",
" <p class="no-responsive__text">\n",
" It looks like your current screen size or device is not yet supported by the WhyLabs Sandbox. The Sandbox is\n",
" best experienced on a desktop computer. Please try maximizing this window or switching to another device. We\n",
" are working on adding support for a larger variety of devices.\n",
" </p>\n",
" </div>\n",
" </div>\n",
" \n",
" </script>\n",
"\n",
" <script src="https://code.jquery.com/jquery-3.6.0.min.js" integrity="sha256-/xUj+3OJU5yExlq6GSYGSHk7tPXikynS7ogEvDej/m4=" crossorigin="anonymous"></script>\n",
"\n",
" <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/6.7.0/d3.min.js" integrity="sha512-cd6CHE+XWDQ33ElJqsi0MdNte3S+bQY819f7p3NUHgwQQLXSKjE4cPZTeGNI+vaxZynk1wVU3hoHmow3m089wA==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>\n",
"\n",
" <script>\n",
" const getReferenceProfile = () => { return {"columns": {"PULocationID": {"histogram": {"start": 41.0, "end": 264.0000264, "width": 0, "counts": [4, 1, 0, 8, 1, 4, 3, 3, 4, 0, 5, 11, 1, 14, 3, 0, 6, 0, 0, 0, 0, 14, 8, 3, 5], "max": 264.0, "min": 41.0, "bins": [41.0, 49.920001056000004, 58.840002112, 67.760003168, 76.680004224, 85.60000528, 94.520006336, 103.440007392, 112.360008448, 121.280009504, 130.20001056, 139.120011616, 148.040012672, 156.960013728, 165.880014784, 174.80001584000001, 183.720016896, 192.640017952, 201.560019008, 210.480020064, 219.40002112000002, 228.320022176, 237.240023232, 246.16002428800002, 255.080025344, 264.0000264], "n": 98}, "frequentItems": null, "drift_from_ref": 0.7806833426960494, "isDiscrete": false, "featureStats": null}, "driver_avg_daily_trips": {"histogram": {"start": 9.0, "end": 44.0000044, "width": 0, "counts": [5, 0, 0, 0, 1, 0, 2, 4, 0, 3, 4, 4, 22, 0, 8, 0, 0, 9, 9, 18, 6, 0, 0, 0, 3], "max": 44.0, "min": 9.0, "bins": [9.0, 10.400000176, 11.800000352, 13.200000528, 14.600000704, 16.00000088, 17.400001056, 18.800001232, 20.200001408, 21.600001584, 23.00000176, 24.400001936000002, 25.800002112, 27.200002288, 28.600002464, 30.00000264, 31.400002816, 32.800002992, 34.200003168, 35.600003344, 37.00000352, 38.400003696, 39.800003872000005, 41.200004048000004, 42.600004224, 44.0000044], "n": 98}, "frequentItems": null, "drift_from_ref": 1.1961979194048912e-44, "isDiscrete": false, "featureStats": null}, "driver_avg_speed": {"histogram": {"start": 13.210000038146973, "end": 31.140002503648375, "width": 0, "counts": [1, 0, 7, 3, 0, 2, 9, 7, 2, 8, 13, 10, 7, 0, 2, 9, 4, 5, 0, 3, 4, 0, 0, 0, 2], "max": 31.139999389648438, "min": 13.210000038146973, "bins": [13.210000038146973, 13.927200136767029, 14.644400235387085, 15.361600334007141, 16.078800432627197, 16.796000531247252, 17.51320062986731, 18.230400728487364, 18.947600827107422, 19.664800925727477, 20.382001024347534, 21.09920112296759, 21.816401221587647, 22.5336013202077, 23.250801418827756, 23.968001517447814, 24.68520161606787, 25.402401714687926, 26.11960181330798, 26.83680191192804, 27.554002010548096, 28.27120210916815, 28.988402207788205, 29.705602306408263, 30.422802405028317, 31.140002503648375], "n": 98}, "frequentItems": null, "drift_from_ref": 0.8014866324598566, "isDiscrete": false, "featureStats": null}, "driver_rate_1m": {"histogram": {"start": 1.0, "end": 4.0000004, "width": 0, "counts": [10, 0, 0, 0, 0, 0, 0, 0, 34, 0, 0, 0, 0, 0, 0, 0, 45, 0, 0, 0, 0, 0, 0, 0, 9], "max": 4.0, "min": 1.0, "bins": [1.0, 1.120000016, 1.240000032, 1.360000048, 1.480000064, 1.60000008, 1.7200000960000001, 1.8400001120000002, 1.9600001280000001, 2.0800001440000004, 2.20000016, 2.320000176, 2.4400001920000003, 2.560000208, 2.6800002240000005, 2.80000024, 2.9200002560000002, 3.0400002720000003, 3.1600002880000004, 3.280000304, 3.40000032, 3.5200003360000003, 3.6400003520000004, 3.7600003680000005, 3.8800003840000006, 4.0000004], "n": 98}, "frequentItems": null, "drift_from_ref": 7.115285500201713e-07, "isDiscrete": false, "featureStats": null}, "passenger_count": {"histogram": {"start": 0.0, "end": 6.0000006, "width": 0, "counts": [3, 0, 0, 0, 77, 0, 0, 0, 9, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 4], "max": 6.0, "min": 0.0, "bins": [0.0, 0.240000024, 0.480000048, 0.720000072, 0.960000096, 1.2000001200000001, 1.440000144, 1.680000168, 1.920000192, 2.1600002160000003, 2.4000002400000002, 2.640000264, 2.880000288, 3.120000312, 3.360000336, 3.60000036, 3.840000384, 4.080000408, 4.3200004320000005, 4.560000456, 4.8000004800000005, 5.040000504, 5.280000528, 5.520000552, 5.760000576, 6.0000006], "n": 98}, "frequentItems": null, "drift_from_ref": 0.740270502533207, "isDiscrete": false, "featureStats": null}, "pickup_weekday": {"histogram": {"start": 0.0, "end": 0.0, "width": 0, "counts": [98], "max": 0.0, "min": 0.0, "bins": [0.0, 0.0], "n": 98}, "frequentItems": null, "drift_from_ref": 0.0, "isDiscrete": false, "featureStats": null}, "trip_distance": {"histogram": {"start": 0.24, "end": 20.220002022, "width": 0, "counts": [31, 24, 14, 8, 4, 5, 1, 5, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1], "max": 20.22, "min": 0.24, "bins": [0.24, 1.0392000808800002, 1.83840016176, 2.6376002426400005, 3.43680032352, 4.2360004044, 5.035200485280001, 5.83440056616, 6.633600647040001, 7.432800727920001, 8.2320008088, 9.031200889680001, 9.830400970560001, 10.629601051440002, 11.42880113232, 12.2280012132, 13.027201294080001, 13.826401374960001, 14.625601455840002, 15.42480153672, 16.2240016176, 17.02320169848, 17.82240177936, 18.62160186024, 19.42080194112, 20.220002022], "n": 98}, "frequentItems": null, "drift_from_ref": 0.9499654244252739, "isDiscrete": false, "featureStats": null}}, "properties": {"observations": 574, "missing_cells": 0, "missing_percentage": 0.0}} }\n",
" const referenceProfile = getReferenceProfile()\n",
"\n",
" function fixNumberTo(number, decimals = 3) {\n",
" const fractionalDigits = String(number % 1).split('').slice(2, 2 + decimals).join('')\n",
" return `${Math.trunc(number)}.${fractionalDigits}`\n",
" }\n",
"\n",
" function registerHandlebarHelperFunctions() {\n",
" //helper fun\n",
"\n",
" class GenerateChartParams {\n",
" constructor(height, width, data, bottomMargin=20, topMargin=5) {\n",
" this.MARGIN = {\n",
" TOP: topMargin,\n",
" RIGHT: 5,\n",
" BOTTOM: bottomMargin,\n",
" LEFT: 55,\n",
" };\n",
" this.SVG_WIDTH = width;\n",
" this.SVG_HEIGHT = height;\n",
" this.CHART_WIDTH = this.SVG_WIDTH - this.MARGIN.LEFT - this.MARGIN.RIGHT;\n",
" this.CHART_HEIGHT = this.SVG_HEIGHT - this.MARGIN.TOP - this.MARGIN.BOTTOM;\n",
" this.svgEl = d3.create("svg")\n",
" .attr("preserveAspectRatio", "xMinYMin meet")\n",
" .attr("viewBox", "30 0 240 400")\n",
" .classed("svg-content-responsive", true)\n",
" this.maxYValue = d3.max(data, (d) => Math.abs(d.axisY));\n",
" this.xScale = d3\n",
" .scaleBand()\n",
" .domain(data.map((d) => d.axisX))\n",
" .range([this.MARGIN.LEFT, this.MARGIN.LEFT + this.CHART_WIDTH]);\n",
" this.yScale = d3\n",
" .scaleLinear()\n",
" .domain([0, this.maxYValue * 1.02])\n",
" .range([this.CHART_HEIGHT, 0]);\n",
" }\n",
" }\n",
"\n",
" function generateChart(data, height = 70, width = 250, index = 0, referenceProfileExist = false) {\n",
" const sizes = new GenerateChartParams(height, width, data, 5)\n",
" const {\n",
" MARGIN,\n",
" SVG_WIDTH,\n",
" SVG_HEIGHT,\n",
" CHART_WIDTH,\n",
" CHART_HEIGHT,\n",
" svgEl,\n",
" maxYValue,\n",
" xScale,\n",
" yScale\n",
" } = sizes\n",
" const rectColors = ["#44C0E7", "#F5843C"]\n",
"\n",
" // Add the y Axis\n",
" if (!referenceProfileExist) {\n",
" svgEl\n",
" .append("g")\n",
" .attr("transform", "translate(" + MARGIN.LEFT + ", " + MARGIN.TOP + ")")\n",
" .call(d3.axisLeft(yScale).tickValues([0, maxYValue/2, maxYValue]))\n",
" .selectAll("text")\n",
" .style("font-size", "8")\n",
" }\n",
"\n",
" const gChart = svgEl.append("g");\n",
" gChart\n",
" .attr("transform", "translate(" + 20 + ", " + 0 + ")")\n",
" .selectAll(".bar")\n",
" .data(data)\n",
" .enter()\n",
" .append("rect")\n",
" .classed("bar", true)\n",
" .attr("width", xScale.bandwidth() - 1)\n",
" .attr("height", (d) => CHART_HEIGHT - yScale(d.axisY))\n",
" .attr("x", (d) => xScale(d.axisX))\n",
" .attr("y", (d) => yScale(d.axisY) + MARGIN.TOP)\n",
" .attr("fill", rectColors[index]);\n",
"\n",
" return svgEl._groups[0][0].outerHTML;\n",
" }\n",
"\n",
" function chartData(column) {\n",
" const data = [];\n",
" if (column.histogram) {\n",
" column.histogram.counts.slice(0, 30).forEach((count, index) => {\n",
" data.push({\n",
" axisY: count,\n",
" axisX: index,\n",
" });\n",
" });\n",
" } else if (column.frequentItems) {\n",
" Object.entries(column.frequentItems).forEach(([key, {value, estimate}], index) => {\n",
" data.push({\n",
" axisY: estimate,\n",
" axisX: value,\n",
" });\n",
" });\n",
" }\n",
"\n",
" return data\n",
" }\n",
"\n",
" function graph(column, key, referenceColumn) {\n",
" let data = [];\n",
" const columnKey = key.data.key\n",
" let chartValue = false\n",
" let chartColor = undefined\n",
" if(!!referenceColumn){\n",
" column = referenceColumn.columns[columnKey]\n",
" chartValue = true\n",
" chartColor = 1\n",
" } else if (referenceColumn === undefined) {\n",
" column = ""\n",
" }\n",
"\n",
" if (column.histogram || column.frequentItems) {\n",
" data = chartData(column)\n",
" } else if (referenceColumn === null ) {\n",
" $(".svg-container").css("padding-bottom", "0")\n",
" return '<span class="wl-table-cell__bedge-wrap">No data to show the chart</span>';\n",
" } else if (referenceColumn === undefined) {\n",
" $(document).ready(function() {\n",
" $(".reference-table-head").addClass("d-none")\n",
" });\n",
" return ''\n",
" } else if (referenceColumn.frequentItems === undefined){\n",
" return '';\n",
" }\n",
" return `\n",
" <div class="svg-container">\n",
" ${generateChart(data, ...[,,], chartColor, chartValue)}\n",
" </div>\n",
" `;\n",
" }\n",
"\n",
" function formatLabelDate(timestamp) {\n",
" const date = new Date(timestamp);\n",
" const format = d3.timeFormat("%Y-%m-%d %I:%M:%S %p %Z");\n",
" return format(date);\n",
" }\n",
"\n",
" const driftCountElement = (driftCount, driftColor, driftName, driftRange) => `\n",
" <div class="display-flex flex-direction-column">\n",
" <div class="drift-detection-info-drifts-item display-flex align-items">\n",
" <div class="drift-detection-info-drifts-item-text display-flex align-items">\n",
" <p class="drift-detection-info-drifts-item-count mb-0">${driftCount}</p>\n",
" <p class="drift-detection-info-drifts-item-range display-flex justify-content-center mb-0">${driftRange}</p>\n",
" </div>\n",
" </div>\n",
" </div>\n",
" `\n",
"\n",
" const drifts = {\n",
" severe: {\n",
" count: 0,\n",
" range: "with detected drift (0.00 - 0.05)",\n",
" },\n",
" moderate: {\n",
" count: 0,\n",
" range: "with possible drift (0.05 - 0.15)",\n",
" },\n",
" mild: {\n",
" count: 0,\n",
" range: "with no evidence of drift (0.15 - 1.0)",\n",
" }\n",
" };\n",
"\n",
" const countOfDrifts = (driftNumber) => {\n",
" if(driftNumber >= 0 && driftNumber <= 0.05) {\n",
" Object.values(drifts)[0].count++\n",
" return 0\n",
" } else if(driftNumber > 0.05 && driftNumber <= 0.15) {\n",
" Object.values(drifts)[1].count++\n",
" return 1\n",
" } else if(driftNumber > 0.15) {\n",
" Object.values(drifts)[2].count++\n",
" return 2\n",
" }\n",
"\n",
" }\n",
"\n",
" abbreviate_number = function(value, fixed = 0) {\n",
" value = +value\n",
" if (value === null) { return null; } // terminate early\n",
" if (value === 0) { return '0'; } // terminate early\n",
" fixed = (!fixed || fixed < 0) ? 0 : fixed; // number of decimal places to show\n",
" var b = (value).toPrecision(2).split("e"), // get power\n",
" k = b.length === 1 ? 0 : Math.floor(Math.min(b[1].slice(1), 14) / 3), // floor at decimals, ceiling at trillions\n",
" c = k < 1 ? value.toFixed(0 + fixed) : (value / Math.pow(10, k * 3) ).toFixed(1 + fixed), // divide by power\n",
" d = c < 0 ? c : Math.abs(c), // enforce -0 is 0\n",
" newValue = d,\n",
" suffixe = ['', 'K', 'M', 'B', 'T'][k]; // append power\n",
" return {value, newValue, suffixe};\n",
" }\n",
"\n",
" function formatBytes(bytes, decimals = 2) {\n",
" let newValue,\n",
" suffixe = ""\n",
" if (bytes === 0) return '0 Bytes';\n",
"\n",
" const k = 1024;\n",
" const dm = decimals < 0 ? 0 : decimals;\n",
" const sizes = ['Bytes', 'KiB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'];\n",
"\n",
" const i = Math.floor(Math.log(bytes) / Math.log(k));\n",
" newValue = parseFloat((bytes / Math.pow(k, i)).toFixed(dm));\n",
" suffixe = sizes[i]\n",
" return {bytes, newValue, suffixe};\n",
" }\n",
"\n",
" const valueSuffixe = (newNumber) => `\n",
" <div class="statistic-measurement">\n",
" ${newNumber}\n",
" </div>`\n",
" const valueNumber = (newNumber) => `\n",
" <div class="statistic-number">\n",
" ${newNumber}\n",
" </div>`\n",
"\n",
" const numberWithSuffixe = (number, newNumber, suffixe) =>\n",
" `<div class="tooltip-full-number">\n",
" <div class="statistic-number">\n",
" ${newNumber}\n",
" <div class="statistic-measurement">${suffixe}</div>\n",
" </div>\n",
" <span class="tooltiptext">${number}</span>\n",
" </div>`\n",
"\n",
" Handlebars.registerHelper("observations", function (properties) {\n",
" const {value, newValue, suffixe} = abbreviate_number(referenceProfile.properties.observations)\n",
" return numberWithSuffixe(value, valueNumber(newValue), valueNumber(suffixe));\n",
" });\n",
"\n",
" Handlebars.registerHelper("missingCells", function (properties) {\n",
" const {value, newValue, suffixe} = abbreviate_number(referenceProfile.properties.missing_cells)\n",
" if (typeof value !== 'undefined' && typeof newValue !== 'undefined' && typeof suffixe !== 'undefined' ) {\n",
" return numberWithSuffixe(value, valueSuffixe(`${newValue}`), suffixe);\n",
" }\n",
" return numberWithSuffixe(0, valueNumber(0), valueNumber(''));\n",
" });\n",
"\n",
" Handlebars.registerHelper("missingCellsPercentage", function (properties) {\n",
" const {value, newValue, suffixe} = abbreviate_number(referenceProfile.properties.missing_percentage)\n",
" if (typeof value !== 'undefined' && typeof newValue !== 'undefined' && typeof suffixe !== 'undefined' ) {\n",
" return numberWithSuffixe(value, valueSuffixe(`(${newValue}%)`), suffixe);\n",
" }\n",
" return numberWithSuffixe(0, valueSuffixe(`(${0}%)`), '');\n",
" });\n",
"\n",
" Handlebars.registerHelper("getProfileTimeStamp", function (properties) {\n",
" return formatLabelDate(+properties.properties.dataTimestamp)\n",
" });\n",
"\n",
" Handlebars.registerHelper("getProfileName", function (properties) {\n",
" return properties.properties.tags.name\n",
" });\n",
"\n",
" let driftCount = 0;\n",
" const diffFromRefTableElement = (driftFromRefNumber, circleColor) => `\n",
" <div class="text-color padding-5 text-align-end justify-content-center">\n",
" ${driftFromRefNumber}\n",
" </div>\n",
" `\n",
"\n",
" const cheqValueTypeNumber = (profile, profileValue) => {\n",
" let validValue;\n",
" if (profile && profileValue !== undefined && typeof profileValue === "number") {\n",
" return true\n",
" } else if (profileValue !== undefined && typeof profileValue !== "number") {\n",
" return false\n",
" }\n",
" }\n",
"\n",
"\n",
" Handlebars.registerHelper("getpvalue", function (column, key) {\n",
" const columnKey = key.data.key\n",
" const {drift_from_ref} = referenceProfile.columns[columnKey]\n",
"\n",
" if (cheqValueTypeNumber(referenceProfile, drift_from_ref)) {\n",
" const driftFromRefNumber = drift_from_ref.toPrecision(2)\n",
" return Number(driftFromRefNumber)\n",
"\n",
" }\n",
" });\n",
"\n",
" Handlebars.registerHelper("getDiffFromRef", function (column, key) {\n",
" const columnKey = key.data.key\n",
" const {drift_from_ref} = referenceProfile.columns[columnKey]\n",
"\n",
" if (cheqValueTypeNumber(referenceProfile, drift_from_ref)) {\n",
" const driftFromRefNumber = drift_from_ref.toPrecision(2)\n",
" const driftCategory = countOfDrifts(driftFromRefNumber)\n",
" const circleColor = Object.values(drifts)[driftCategory].colorClass\n",
" if (driftCategory==0 || driftCategory == 1){\n",
" driftCount++\n",
" }\n",
" $(document).ready(() => {\n",
" $(".drift-count").html(driftCount)\n",
" $(".all-features").html(Object.keys(referenceProfile.columns).length)\n",
" })\n",
"\n",
" return diffFromRefTableElement(driftFromRefNumber, circleColor)\n",
"\n",
" } else if (cheqValueTypeNumber(referenceProfile, drift_from_ref) !== undefined) {\n",
" Object.values(drifts)[0].count++\n",
" return diffFromRefTableElement("undefined", "severe-drift-circle-color")\n",
" }\n",
"\n",
" return '<p style="color: black">-</p>';\n",
" });\n",
"\n",
" $(document).ready(() =>\n",
" Object.values(drifts).map(({count, name, colorClass, range}) =>{\n",
" $("#drift-detection-info-drifts")\n",
" .append(driftCountElement(count, colorClass, name, range))\n",
" })\n",
" )\n",
"\n",
" Handlebars.registerHelper("inferredType", function (column) {\n",
" let infferedType = "";\n",
"\n",
" if (column.isDiscrete) {\n",
" infferedType = "Discrete";\n",
" } else {\n",
" infferedType = "Non-discrete";\n",
" }\n",
" return infferedType;\n",
" });\n",
"\n",
" Handlebars.registerHelper("frequentItems", function (column) {\n",
" frequentItemsElemString = "";\n",
" if (column.isDiscrete) {\n",
" const slicedFrequentItems = column.frequentItems.items.slice(0, 5);\n",
" for (let fi = 0; fi < slicedFrequentItems.length; fi++) {\n",
" frequentItemsElemString +=\n",
" '<span class="wl-table-cell__bedge">' + slicedFrequentItems[fi].jsonValue + "</span>";\n",
" }\n",
" } else {\n",
" frequentItemsElemString += "No data to show";\n",
" }\n",
" return frequentItemsElemString;\n",
" });\n",
"\n",
" Handlebars.registerHelper("totalCount", function (column) {\n",
" if (column.featureStats) {\n",
" return column.featureStats.total_count;\n",
" }\n",
"\n",
" return "-";\n",
" });\n",
"\n",
" Handlebars.registerHelper("mean", function (column) {\n",
" if (column.featureStats?.descriptive_statistics) {\n",
" return fixNumberTo(column.featureStats.descriptive_statistics.mean);\n",
" }\n",
" return "-";\n",
" });\n",
"\n",
" Handlebars.registerHelper("getGraphHtml",(column,key) => graph(column, key, null));\n",
" Handlebars.registerHelper("getReferenceGraphHtml",(column,key) => graph(column, key, referenceProfile));\n",
"\n",
" Handlebars.registerHelper("getDiscreteTypeCount", function () {\n",
" let count = 0;\n",
"\n",
" Object.entries(this.columns).forEach((feature) => {\n",
" if (feature[1].isDiscrete === true) {\n",
" count++;\n",
" }\n",
" });\n",
" return count.toString();\n",
" });\n",
"\n",
" Handlebars.registerHelper("getNonDiscreteTypeCount", function () {\n",
" let count = 0;\n",
"\n",
" Object.entries(this.columns).forEach((feature) => {\n",
" if (feature[1].isDiscrete === false) {\n",
" count++;\n",
" }\n",
" });\n",
" return count;\n",
" });\n",
"\n",
" Handlebars.registerHelper("getUnknownTypeCount", function () {\n",
" let count = 0;\n",
" return count;\n",
" Object.entries(this.columns).forEach((feature) => {\n",
" if (!feature[1].isDiscrete) {\n",
" count++;\n",
" }\n",
" });\n",
" return count;\n",
" });\n",
" }\n",
"\n",
" function initHandlebarsTemplate() {\n",
" // Replace this context with JSON from .py file\n",
" const context = {"columns": {"PULocationID": {"histogram": {"start": 13.0, "end": 262.0000262, "width": 0, "counts": [3, 0, 2, 1, 3, 3, 5, 3, 3, 0, 10, 6, 13, 4, 4, 0, 1, 1, 11, 5, 4], "max": 262.0, "min": 13.0, "bins": [13.0, 24.857144104761904, 36.71428820952381, 48.571432314285715, 60.428576419047616, 72.28572052380952, 84.14286462857143, 96.00000873333333, 107.85715283809523, 119.71429694285713, 131.57144104761903, 143.42858515238095, 155.28572925714286, 167.14287336190475, 179.00001746666666, 190.85716157142855, 202.71430567619046, 214.57144978095238, 226.42859388571426, 238.28573799047618, 250.14288209523806, 262.0000262], "n": 82}, "frequentItems": null, "drift_from_ref": null, "isDiscrete": false, "featureStats": {"total_count": 82, "missing": 0, "distinct": 51.219517410529, "min": 13.0, "max": 262.0, "range": 249.0, "quantile_statistics": {"fifth_percentile": 48.0, "iqr": 102.0, "q1": 113.0, "median": 158.0, "q3": 215.0, "ninety_fifth_percentile": 249.0}, "descriptive_statistics": {"stddev": 64.79287756255695, "coefficient_of_variation": 0.415891660284123, "sum": 12774.999999999996, "variance": 4198.1169828364955, "mean": 155.79268292682923}}}, "driver_avg_daily_trips": {"histogram": {"start": 0.0, "end": 20.000002, "width": 0, "counts": [20, 5, 1, 12, 7, 9, 7, 2, 3, 0, 0, 0, 4, 4, 0, 0, 2, 0, 5, 0, 1], "max": 20.0, "min": 0.0, "bins": [0.0, 0.9523810476190475, 1.904762095238095, 2.8571431428571428, 3.80952419047619, 4.761905238095237, 5.7142862857142855, 6.666667333333333, 7.61904838095238, 8.571429428571427, 9.523810476190475, 10.476191523809522, 11.428572571428571, 12.380953619047618, 13.333334666666666, 14.285715714285713, 15.23809676190476, 16.19047780952381, 17.142858857142855, 18.095239904761904, 19.04762095238095, 20.000002], "n": 82}, "frequentItems": null, "drift_from_ref": null, "isDiscrete": false, "featureStats": {"total_count": 82, "missing": 0, "distinct": 17.073171282929152, "min": 0.0, "max": 20.0, "range": 20.0, "quantile_statistics": {"fifth_percentile": 0.0, "iqr": 6.0, "q1": 1.0, "median": 4.0, "q3": 7.0, "ninety_fifth_percentile": 18.0}, "descriptive_statistics": {"stddev": 5.488977759503495, "coefficient_of_variation": 1.0276168408202893, "sum": 437.9999999999999, "variance": 30.128876844324004, "mean": 5.341463414634145}}}, "driver_avg_speed": {"histogram": {"start": 12.069999694824219, "end": 26.080002531706047, "width": 0, "counts": [3, 0, 2, 0, 0, 2, 3, 1, 7, 11, 7, 0, 5, 1, 6, 3, 11, 4, 0, 3, 13], "max": 26.079999923706055, "min": 12.069999694824219, "bins": [12.069999694824219, 12.737142687056688, 13.404285679289154, 14.071428671521623, 14.73857166375409, 15.405714655986559, 16.072857648219028, 16.740000640451495, 17.40714363268396, 18.074286624916432, 18.7414296171489, 19.408572609381366, 20.075715601613837, 20.7428585938463, 21.41000158607877, 22.077144578311238, 22.744287570543705, 23.411430562776175, 24.078573555008642, 24.74571654724111, 25.41285953947358, 26.080002531706047], "n": 82}, "frequentItems": null, "drift_from_ref": null, "isDiscrete": false, "featureStats": {"total_count": 82, "missing": 0, "distinct": 42.6829304334113, "min": 12.069999694824219, "max": 26.079999923706055, "range": 14.010000228881836, "quantile_statistics": {"fifth_percentile": 13.460000038146973, "iqr": 4.970001220703125, "q1": 18.1299991607666, "median": 20.899999618530273, "q3": 23.100000381469727, "ninety_fifth_percentile": 25.8700008392334}, "descriptive_statistics": {"stddev": 3.678358010938584, "coefficient_of_variation": 0.1773348833364555, "sum": 1700.8800029754634, "variance": 13.530317656636058, "mean": 20.742439060676382}}}, "driver_rate_1m": {"histogram": {"start": 1.0, "end": 10.000001, "width": 0, "counts": [16, 0, 4, 0, 22, 0, 11, 0, 0, 15, 0, 8, 0, 3, 0, 0, 0, 0, 2, 0, 1], "max": 10.0, "min": 1.0, "bins": [1.0, 1.4285714761904762, 1.8571429523809524, 2.2857144285714286, 2.714285904761905, 3.1428573809523805, 3.5714288571428567, 4.0000003333333325, 4.42857180952381, 4.857143285714285, 5.285714761904761, 5.714286238095237, 6.1428577142857135, 6.57142919047619, 7.000000666666666, 7.428572142857142, 7.857143619047618, 8.285715095238094, 8.71428657142857, 9.142858047619047, 9.571429523809522, 10.000001], "n": 82}, "frequentItems": null, "drift_from_ref": null, "isDiscrete": false, "featureStats": {"total_count": 82, "missing": 0, "distinct": 10.975609974163339, "min": 1.0, "max": 10.0, "range": 9.0, "quantile_statistics": {"fifth_percentile": 1.0, "iqr": 2.0, "q1": 3.0, "median": 3.0, "q3": 5.0, "ninety_fifth_percentile": 7.0}, "descriptive_statistics": {"stddev": 2.0369959756065197, "coefficient_of_variation": 0.5458616666657994, "sum": 306.0, "variance": 4.149352604637158, "mean": 3.7317073170731705}}}, "passenger_count": {"histogram": {"start": 1.0, "end": 3.0000003, "width": 0, "counts": [75, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3], "max": 3.0, "min": 1.0, "bins": [1.0, 1.0952381095238095, 1.190476219047619, 1.2857143285714285, 1.380952438095238, 1.4761905476190476, 1.571428657142857, 1.6666667666666668, 1.761904876190476, 1.8571429857142858, 1.952381095238095, 2.047619204761905, 2.142857314285714, 2.238095423809524, 2.3333335333333336, 2.428571642857143, 2.523809752380952, 2.619047861904762, 2.7142859714285716, 2.809524080952381, 2.90476219047619, 3.0000003], "n": 82}, "frequentItems": null, "drift_from_ref": null, "isDiscrete": false, "featureStats": {"total_count": 82, "missing": 0, "distinct": 3.658536603538001, "min": 1.0, "max": 3.0, "range": 2.0, "quantile_statistics": {"fifth_percentile": 1.0, "iqr": 0.0, "q1": 1.0, "median": 1.0, "q3": 1.0, "ninety_fifth_percentile": 2.0}, "descriptive_statistics": {"stddev": 0.4271711110139755, "coefficient_of_variation": 0.38073946851245627, "sum": 92.00000000000003, "variance": 0.18247515808491416, "mean": 1.1219512195121955}}}, "pickup_weekday": {"histogram": {"start": 6.0, "end": 6.0000006, "width": 0, "counts": [82], "max": 6.0, "min": 6.0, "bins": [6.0, 6.0000006], "n": 82}, "frequentItems": null, "drift_from_ref": null, "isDiscrete": false, "featureStats": {"total_count": 82, "missing": 0, "distinct": 1.2195121951219512, "min": 6.0, "max": 6.0, "range": 0.0, "quantile_statistics": {"fifth_percentile": 6.0, "iqr": 0.0, "q1": 6.0, "median": 6.0, "q3": 6.0, "ninety_fifth_percentile": 6.0}, "descriptive_statistics": {"stddev": 0.0, "coefficient_of_variation": 0.0, "sum": 492.0, "variance": 0.0, "mean": 6.0}}}, "trip_distance": {"histogram": {"start": 0.28, "end": 19.760001976, "width": 0, "counts": [33, 14, 8, 8, 6, 2, 3, 2, 0, 0, 0, 1, 0, 1, 0, 2, 0, 0, 1, 0, 1], "max": 19.76, "min": 0.28, "bins": [0.28, 1.2076191417142859, 2.1352382834285715, 3.062857425142857, 3.9904765668571427, 4.918095708571429, 5.845714850285715, 6.773333992, 7.700953133714286, 8.62857227542857, 9.556191417142857, 10.483810558857142, 11.411429700571428, 12.339048842285713, 13.266667984, 14.194287125714284, 15.121906267428571, 16.04952540914286, 16.977144550857144, 17.90476369257143, 18.832382834285717, 19.760001976], "n": 82}, "frequentItems": null, "drift_from_ref": null, "isDiscrete": false, "featureStats": {"total_count": 82, "missing": 0, "distinct": 89.02440616270721, "min": 0.28, "max": 19.76, "range": 19.48, "quantile_statistics": {"fifth_percentile": 0.47, "iqr": 3.16, "q1": 0.76, "median": 1.76, "q3": 3.92, "ninety_fifth_percentile": 13.25}, "descriptive_statistics": {"stddev": 3.8729128287352705, "coefficient_of_variation": 1.2376416677953714, "sum": 256.5999999999999, "variance": 14.999453778982236, "mean": 3.1292682926829256}}}}, "properties": null};\n",
" // Config handlebars and pass data to HBS template\n",
" const source = document.getElementById("entry-template").innerHTML;\n",
" const template = Handlebars.compile(source);\n",
" const html = template(context);\n",
" const target = document.getElementById("generated-html");\n",
" target.innerHTML = html;\n",
" }\n",
"\n",
" function initWebsiteScripts() {\n",
" const $featureSearch = document.getElementById("wl__feature-search");\n",
" const $tableBody = document.getElementById("table-body");\n",
" const $discrete = document.getElementById("inferredDiscrete");\n",
" const $nonDiscrete = document.getElementById("inferredNonDiscrete");\n",
" const $unknown = document.getElementById("inferredUnknown");\n",
" const $diffFromRef = document.getElementById("diff-from-ref");\n",
"\n",
" const activeTypes = {\n",
" discrete: true,\n",
" "non-discrete": true,\n",
" unknown: true,\n",
" };\n",
"\n",
" var driftOrder = {\n",
" original: true,\n",
" descending: false,\n",
" ascending: false,\n",
" };\n",
" var originalColumnOrder;\n",
"\n",
" let searchString = "";\n",
"\n",
" function debounce(func, wait, immediate) {\n",
" let timeout;\n",
"\n",
" return function () {\n",
" const context = this;\n",
" const args = arguments;\n",
" const later = function () {\n",
" timeout = null;\n",
" if (!immediate) func.apply(context, args);\n",
" };\n",
"\n",
" const callNow = immediate && !timeout;\n",
" clearTimeout(timeout);\n",
" timeout = setTimeout(later, wait);\n",
" if (callNow) func.apply(context, args);\n",
" };\n",
" }\n",
"\n",
" function filterNotification() {\n",
" const $notifCircleContainer = $(".notif-circle-container")\n",
" const $boxes = $('.wl_filter-options>.form-check>input[name=checkbox]:checked');\n",
" const item = Object.values($boxes).find(function(value) { return $(value)[0] === undefined});\n",
" if (item === undefined) {\n",
" $notifCircleContainer.removeClass("d-none")\n",
" } else {\n",
" $notifCircleContainer.addClass("d-none")\n",
" }\n",
" }\n",
"\n",
"\n",
" function orderByDrift(direction) {\n",
" var tableBodyChildren = $tableBody.children;\n",
" if (driftOrder['original']) {\n",
" driftOrder['original'] = false\n",
" driftOrder['descending'] = true\n",
"\n",
" originalColumnOrder = [...tableBodyChildren];\n",
" tableBodyChildren = [...tableBodyChildren].sort((a,b) => (b.dataset.pValue - a.dataset.pValue))\n",
" document.getElementById("pvalue").style.display = "none";\n",
" document.getElementById("pvalue-desc").style.display = "";\n",
"\n",
" } else if (driftOrder['descending']) {\n",
" driftOrder['descending'] = false\n",
" driftOrder['ascending'] = true\n",
" tableBodyChildren = [...tableBodyChildren].sort((a,b) => (a.dataset.pValue - b.dataset.pValue))\n",
" document.getElementById("pvalue-desc").style.display = "none";\n",
" document.getElementById("pvalue-asc").style.display = "";\n",
"\n",
"\n",
" } else if (driftOrder['ascending']) {\n",
" driftOrder['ascending'] = false\n",
" driftOrder['original'] = true\n",
"\n",
" tableBodyChildren = originalColumnOrder;\n",
" document.getElementById("pvalue-asc").style.display = "none";\n",
" document.getElementById("pvalue").style.display = "";\n",
"\n",
" }\n",
"\n",
" for (let i = 0; i < tableBodyChildren.length; i++) {\n",
" console.log(tableBodyChildren[i].dataset.pValue)\n",
" }\n",
" for (var i = 0; i < tableBodyChildren.length; i++){$tableBody.append(tableBodyChildren[i])}\n",
" }\n",
"\n",
"\n",
" function handleSearch() {\n",
" const tableBodyChildren = $tableBody.children;\n",
"\n",
" for (let i = 0; i < tableBodyChildren.length; i++) {\n",
" const type = tableBodyChildren[i].dataset.inferredType.toLowerCase();\n",
" const name = tableBodyChildren[i].dataset.featureName.toLowerCase();\n",
" if (activeTypes[type] && name.startsWith(searchString)) {\n",
" tableBodyChildren[i].style.display = "";\n",
" } else {\n",
" tableBodyChildren[i].style.display = "none";\n",
" }\n",
" }\n",
" }\n",
"\n",
" $featureSearch.addEventListener(\n",
" "keyup",\n",
" debounce((event) => {\n",
" searchString = event.target.value.toLowerCase();\n",
" handleSearch();\n",
" }, 100),\n",
" );\n",
"\n",
" $diffFromRef.addEventListener(\n",
" "click",\n",
" debounce((event) => {\n",
" orderByDrift();\n",
" }, 100),\n",
" );\n",
"\n",
" $discrete.addEventListener("change", (event) => {\n",
" if (event.currentTarget.checked) {\n",
" activeTypes["discrete"] = true;\n",
" } else {\n",
" activeTypes["discrete"] = false;\n",
" }\n",
" handleSearch();\n",
" filterNotification()\n",
" });\n",
"\n",
" $nonDiscrete.addEventListener("change", (event) => {\n",
" if (event.currentTarget.checked) {\n",
" activeTypes["non-discrete"] = true;\n",
" } else {\n",
" activeTypes["non-discrete"] = false;\n",
" }\n",
" handleSearch();\n",
" filterNotification()\n",
" });\n",
"\n",
" $unknown.addEventListener("change", (event) => {\n",
" if (event.currentTarget.checked) {\n",
" activeTypes["unknown"] = true;\n",
" } else {\n",
" activeTypes["unknown"] = false;\n",
" }\n",
" handleSearch();\n",
" filterNotification()\n",
" });\n",
"\n",
" function checkCurrentProfile(item, referenceItem) {\n",
" if (referenceProfile && Object.values(referenceProfile)) {\n",
" return referenceItem\n",
" } else {\n",
" return item\n",
" }\n",
" }\n",
"\n",
" if(checkCurrentProfile(true, false)) {\n",
" $(".svg-container").css("padding-bottom", "27%")\n",
" $("#diff-from-ref").addClass("d-none")\n",
" $(".diff-from-ref-table-cell").addClass("d-none")\n",
" } else {\n",
" $("#diff-from-ref").removeClass("d-none")\n",
" $(".diff-from-ref-table-cell").removeClass("d-none")\n",
" }\n",
" }\n",
"\n",
" function checkedBoxes() {\n",
" const $boxes = $('input[name=checkbox]:checked');\n",
" const $notifCircleContainer = $(".notif-circle-container")\n",
"\n",
" if ($boxes.length) {\n",
" $notifCircleContainer.removeClass("d-none")\n",
" }\n",
" }\n",
"\n",
" function openFilter() {\n",
" const $filterOptions = $(".dropdown-container");\n",
" const $notifCircleContainer = $(".notif-circle-container")\n",
" const filterClass = $filterOptions.attr("class");\n",
"\n",
" if (filterClass.indexOf("d-none") > 0) {\n",
" $notifCircleContainer.addClass("d-none")\n",
" $filterOptions.removeClass("d-none");\n",
" $(".filter-icon").addClass("d-none")\n",
" $(".close-filter-icon").removeClass("d-none")\n",
" } else {\n",
" $filterOptions.addClass("d-none");\n",
" $(".close-filter-icon").addClass("d-none")\n",
" $(".filter-icon").removeClass("d-none")\n",
" checkedBoxes()\n",
" }\n",
" }\n",
"\n",
" // Invoke functions -- keep in mind invokation order\n",
" registerHandlebarHelperFunctions();\n",
" initHandlebarsTemplate();\n",
" initWebsiteScripts();\n",
" </script>\n",
"</html>\n",
"\" width=100% height=1000px\n",
" frameBorder=0></iframe>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"visualization.summary_drift_report()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The report warns us of 3 possible drifts in the following features:\n",
"\n",
"- driver_avg_daily_trips\n",
"- driver_rate_1m\n",
"- pickup_weekday\n",
"\n",
"Indeed, we artificially changed the first two features, so we might expect to see a drift alert for those. The third one is also a drift, but probably not a relevant one, since it's pretty obvious that a feature that reflects the day of the week will be different for daily batches (unless both of them are from the same day of the week!)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can inspect these features further by using `distribution_chart()` for the driver's rating and `double_histogram()` for the daily trips:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> Note: Even though both features are integers, each feature can be better visualized with different types of visualizations. That happens because integers can be viewed as number, properly, or as a sort of encoding categorical variables. Since the driver's rating has few different possible number, and it wouldn't make sense to group different numbers in a single bin, this feature is better visualized by treating them as categorical variables, and therefore using the `distribution_chart()` visualization."
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div></div><iframe srcdoc=\"<!DOCTYPE html>\n",
"<html lang="en">\n",
" <head>\n",
" <meta charset="UTF-8" />\n",
" <meta http-equiv="X-UA-Compatible" content="IE=edge" />\n",
" <meta name="viewport" content="width=device-width, initial-scale=1.0" />\n",
" <meta name="description" content="" />\n",
" <meta name="author" content="" />\n",
"\n",
" <title>Profile Visualizer | whylogs</title>\n",
"\n",
" <link rel="icon" href="images/whylabs-favicon.png" type="image/png" sizes="16x16" />\n",
" <link rel="preconnect" href="https://fonts.googleapis.com" />\n",
" <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />\n",
" <link href="https://fonts.googleapis.com/css2?family=Asap:wght@400;500;600;700&display=swap" rel="stylesheet" />\n",
" <link rel="preconnect" href="https://fonts.gstatic.com" />\n",
" <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.0/dist/css/bootstrap.min.css" />\n",
"\n",
" <script\n",
" src="https://cdnjs.cloudflare.com/ajax/libs/handlebars.js/4.7.7/handlebars.min.js"\n",
" integrity="sha512-RNLkV3d+aLtfcpEyFG8jRbnWHxUqVZozacROI4J2F1sTaDqo1dPQYs01OMi1t1w9Y2FdbSCDSQ2ZVdAC8bzgAg=="\n",
" crossorigin="anonymous"\n",
" referrerpolicy="no-referrer"\n",
" ></script>\n",
"\n",
" <style type="text/css">\n",
"\n",
" :root {\n",
" /** Branded colors */\n",
" --brandSecondary900: #4f595b;\n",
" --secondaryLight1000: #313b3d;\n",
" /** Purpose colors */\n",
" --tealBackground: #eaf2f3;\n",
" }\n",
"\n",
" /* RESET STYLE */\n",
" *,\n",
" *::after,\n",
" *::before {\n",
" margin: 0;\n",
" padding: 0;\n",
" box-sizing: border-box;\n",
" overflow-y: hidden;\n",
" }\n",
"\n",
" /* Screen on smaller screens */\n",
" .no-responsive {\n",
" display: none;\n",
" position: fixed;\n",
" top: 0;\n",
" left: 0;\n",
" z-index: 1031;\n",
" width: 100vw;\n",
" height: 100vh;\n",
" background-color: var(--tealBackground);\n",
" display: flex;\n",
" align-items: center;\n",
" justify-content: center;\n",
" }\n",
"\n",
" .desktop-content {\n",
" display: flex;\n",
" flex-direction: column;\n",
" }\n",
"\n",
" .no-responsive__content {\n",
" max-width: 600px;\n",
" width: 100%;\n",
" padding: 0 24px;\n",
" }\n",
"\n",
" .no-responsive__title {\n",
" font-size: 96px;\n",
" font-weight: 300;\n",
" color: var(--brandSecondary900);\n",
" line-height: 1.167;\n",
" }\n",
"\n",
" .no-responsive__text {\n",
" margin: 0;\n",
" font-size: 16px;\n",
" font-weight: 400;\n",
" color: var(--brandSecondary900);\n",
" line-height: 1.5;\n",
" }\n",
"\n",
" .svg-container {\n",
" display: inline-block;\n",
" position: relative;\n",
" width: 100%;\n",
" vertical-align: top;\n",
" overflow: hidden;\n",
" }\n",
"\n",
" .svg-content-responsive {\n",
" display: inline-block;\n",
" position: absolute;\n",
" left: 0;\n",
" }\n",
"\n",
" .circle-color {\n",
" display: inline-block;\n",
" padding: 5px;\n",
" border-radius: 50px;\n",
" }\n",
"\n",
" .colors-for-distingushing-charts {\n",
" padding-right: 10px;\n",
" }\n",
"\n",
" .display-flex{\n",
" display: flex;\n",
" }\n",
"\n",
" .flex-direction-column {\n",
" flex-direction: column;\n",
" }\n",
"\n",
" .align-items-flex-end {\n",
" align-items: flex-end;\n",
" }\n",
"\n",
" .chart-box-title {\n",
" width: 98%;\n",
" justify-content: space-between;\n",
" margin: 10px;\n",
" margin-top: 15px;\n",
" bottom: 0;\n",
" }\n",
"\n",
" .chart-box-title p{\n",
" margin-bottom: 0;\n",
" font-family: Asap;\n",
" font-weight: bold;\n",
" font-size: 18px;\n",
" line-height: 16px;\n",
" color: #4F595B;\n",
" }\n",
"\n",
" .error-message {\n",
" display: flex;\n",
" justify-content: center;\n",
" align-items: center;\n",
" color: rgb(255, 114, 71);\n",
" font-size: 30px;\n",
" font-weight: 900;\n",
" }\n",
"\n",
" @media screen and (min-width: 500px) {\n",
" .desktop-content {\n",
" display: block;\n",
" }\n",
" .no-responsive {\n",
" display: none;\n",
" }\n",
" }\n",
" </style>\n",
" </head>\n",
"\n",
" <body id="generated-html"></body>\n",
" <script id="entry-template" type="text/x-handlebars-template">\n",
" \n",
" <div class="desktop-content">\n",
"{{#each this}} <div class="chart-box" id="chart-box">\n",
" <div class="chart-box-title display-flex">\n",
" <p>{{@key}}</p>\n",
" <div class="display-flex">\n",
" <div class="colors-for-distingushing-charts">\n",
" <div class="circle-color" style="background: #44C0E7;"></div>\n",
" <text alignment-baseline="middle" style="font-size: 15px;">Target</text>\n",
" </div>\n",
" <div class="colors-for-distingushing-charts">\n",
" <div class="circle-color" style="background: #F5843C"></div>\n",
" <text alignment-baseline="middle" style="font-size: 15px;">Reference</text>\n",
" </div>\n",
" </div></div>\n",
" <div class="svg-container">\n",
" {{{getDoubleHistogramChart this}}}\n",
" </div>\n",
" </div>\n",
"{{/each}} </div>\n",
" <div class="no-responsive">\n",
" <div class="no-responsive__content">\n",
" <h1 class="no-responsive__title">Hold on! :)</h1>\n",
" <p class="no-responsive__text">\n",
" It looks like your current screen size or device is not yet supported by the WhyLabs Sandbox. The Sandbox is\n",
" best experienced on a desktop computer. Please try maximizing this window or switching to another device. We\n",
" are working on adding support for a larger variety of devices.\n",
" </p>\n",
" </div>\n",
" </div>\n",
" \n",
" </script>\n",
"\n",
" <script src="https://code.jquery.com/jquery-3.6.0.min.js" integrity="sha256-/xUj+3OJU5yExlq6GSYGSHk7tPXikynS7ogEvDej/m4=" crossorigin="anonymous"></script>\n",
"\n",
" <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/6.7.0/d3.min.js" integrity="sha512-cd6CHE+XWDQ33ElJqsi0MdNte3S+bQY819f7p3NUHgwQQLXSKjE4cPZTeGNI+vaxZynk1wVU3hoHmow3m089wA==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>\n",
"\n",
" <script>\n",
" function registerHandlebarHelperFunctions() {\n",
"\n",
" const findAndDeleteUndefined = (axisData) => {\n",
" const undefinedAxisIndex = axisData.findIndex((axis) => axis === undefined)\n",
" if (undefinedAxisIndex == -1) {\n",
" return axisData;\n",
" }\n",
"\n",
" const result = [...axisData.slice(0, undefinedAxisIndex), ...axisData.slice(undefinedAxisIndex + 1)]\n",
" return result\n",
" }\n",
"\n",
" const filterAndSortChartData = (overlappedHistogramData, histogramData) => {\n",
" const filteredData = overlappedHistogramData\n",
" .map((d) =>\n",
" histogramData\n",
" .filter((ref) => d.axisX === ref.axisX )[0])\n",
" .sort((a, b) => {\n",
" if (+a.axisY < +b.axisY) {\n",
" return 1;\n",
" }\n",
" if (+a.axisY > +b.axisY) {\n",
" return -1;\n",
" }\n",
"\n",
" return 0;\n",
" })\n",
" .slice(0, 20)\n",
"\n",
" return findAndDeleteUndefined(filteredData)\n",
" }\n",
"\n",
" class GenerateChartParams {\n",
" constructor(height, width, data, referenceData, bottomMargin=30, topMargin=5) {\n",
" this.MARGIN = {\n",
" TOP: topMargin,\n",
" RIGHT: 5,\n",
" BOTTOM: bottomMargin,\n",
" LEFT: 55,\n",
" };\n",
" this.SVG_WIDTH = width;\n",
" this.SVG_HEIGHT = height;\n",
" this.CHART_WIDTH = this.SVG_WIDTH - this.MARGIN.LEFT - this.MARGIN.RIGHT;\n",
" this.CHART_HEIGHT = this.SVG_HEIGHT - this.MARGIN.TOP - this.MARGIN.BOTTOM;\n",
" this.svgEl = d3.create("svg")\n",
" .attr("preserveAspectRatio", "xMinYMin meet")\n",
" .attr("viewBox", `0 0 ${$(window).width()} ${$(window).height()-30}`)\n",
" .classed("svg-content-responsive", true)\n",
" this.maxYValue = d3.max([...data, ...referenceData], (d) => Math.abs(d.axisY));\n",
" this.xScale = d3\n",
" .scaleBand()\n",
" .domain(filterAndSortChartData(data, referenceData).map((sortedCounts) => sortedCounts?.axisX))\n",
" .range([this.MARGIN.LEFT, this.MARGIN.LEFT + this.CHART_WIDTH]);\n",
" this.yScale = d3\n",
" .scaleLinear()\n",
" .domain([0, this.maxYValue * 1.2])\n",
" .range([this.CHART_HEIGHT, 0]);\n",
" }\n",
" }\n",
"\n",
" function chartData(column) {\n",
" const data = [];\n",
" if (column.frequentItems) {\n",
" Object.entries(column.frequentItems).forEach(([key, {value, estimate}], index) => {\n",
" data.push({\n",
" axisY: estimate,\n",
" axisX: value,\n",
" });\n",
" });\n",
" } else {\n",
" $(document).ready(() =>\n",
" $(".desktop-content").html(`\n",
" <p style="height: ${$(window).height()}px" class="error-message">\n",
" Something went wrong. Please try again.\n",
" </p>\n",
" `)\n",
" )\n",
" }\n",
"\n",
" return data\n",
" }\n",
"\n",
" function generateBarChart(histogramData, overlappedHistogramData) {\n",
" let yFormat,\n",
" xFormat;\n",
" const data = filterAndSortChartData(chartData(histogramData), chartData(overlappedHistogramData)).map((axis, index) => {\n",
" if (axis) {\n",
" const findIndex = chartData(histogramData).findIndex((value) => value.axisX === axis.axisX)\n",
" return {\n",
" group: axis.axisX,\n",
" profile: axis.axisY,\n",
" reference_profile: chartData(histogramData)[findIndex].axisY\n",
" }\n",
" }\n",
" return 0;\n",
" })\n",
"\n",
" const sizes = new GenerateChartParams($(window).height()-60, $(window).width(), chartData(histogramData), chartData(overlappedHistogramData))\n",
" let {\n",
" MARGIN,\n",
" SVG_WIDTH,\n",
" SVG_HEIGHT,\n",
" CHART_WIDTH,\n",
" CHART_HEIGHT,\n",
" svgEl,\n",
" xScale,\n",
" yScale\n",
" } = sizes\n",
"\n",
" const rectColors = ["#44C0E7", "#F5843C"]\n",
" const subgroups = ['reference_profile', 'profile']\n",
"\n",
" xScale.padding([0.3])\n",
"\n",
" const xAxis = d3.axisBottom(xScale).ticks(SVG_WIDTH / 80, xFormat).tickSizeOuter(0);\n",
" const yAxis = d3.axisLeft(yScale).ticks(SVG_HEIGHT / 40, yFormat);\n",
" yFormat = yScale.tickFormat(100, yFormat);\n",
"\n",
" svgEl.append("g")\n",
" .attr("transform", `translate(${MARGIN.LEFT}, 0)`)\n",
" .attr("id", "g1")\n",
" .call(yAxis)\n",
" .call(g => g.select(".domain").remove())\n",
" .call(g => g.selectAll(".tick line")\n",
" .attr("x2", CHART_WIDTH)\n",
" .attr("stroke-opacity", 0.1))\n",
" .call(g => g.append("text")\n",
" .attr("x", -MARGIN.LEFT)\n",
" .attr("y", 10)\n",
" .attr("fill", "currentColor")\n",
" .attr("text-anchor", "start"));\n",
"\n",
" svgEl.append("text")\n",
" .attr("transform",\n",
" "translate(" + (CHART_WIDTH/2) + " ," +\n",
" (CHART_HEIGHT + MARGIN.TOP + 40) + ")")\n",
" .style("text-anchor", "middle")\n",
" .text("Values")\n",
" .style("font-size", "15")\n",
" .style("opacity", "0.6")\n",
"\n",
" svgEl.append("text")\n",
" .attr("transform", "rotate(-90)")\n",
" .attr("y", 0)\n",
" .attr("x", 0 - (SVG_HEIGHT / 2))\n",
" .attr("dy", "1em")\n",
" .style("text-anchor", "middle")\n",
" .text("Counts")\n",
" .style("font-size", "15")\n",
" .style("opacity", "0.6")\n",
"\n",
" svgEl.append("g")\n",
" .attr("transform", `translate(0,${SVG_HEIGHT - MARGIN.BOTTOM})`)\n",
" .attr("id", "g2")\n",
" .call(xAxis)\n",
" .call(g => g.select(".domain").remove())\n",
" .call(g => g.selectAll(".tick line").remove())\n",
" .call(g => g.append("text")\n",
" .attr("x", SVG_WIDTH - MARGIN.RIGHT)\n",
" .attr("y", 27)\n",
" .attr("fill", "currentColor")\n",
" .attr("text-anchor", "end"));\n",
"\n",
" const xSubgroup = d3.scaleBand()\n",
" .domain(subgroups)\n",
" .range([0, xScale.bandwidth()])\n",
"\n",
" const color = d3.scaleOrdinal()\n",
" .domain(subgroups)\n",
" .range(rectColors)\n",
"\n",
" svgEl.append("g")\n",
" .attr("id", "g3")\n",
" .selectAll("g")\n",
" .data(data)\n",
" .enter()\n",
" .append("g")\n",
" .attr("transform", function(d) { return "translate(" + xScale(d?.group) + ",0)"; })\n",
" .attr("id", "g4")\n",
" .selectAll("rect")\n",
" .data(function(d) { return subgroups.map(function(key) { return {key: key, value: d && d[key]}; }); })\n",
" .enter().append("rect")\n",
" .attr("x", function(d) { return xSubgroup(d.key); })\n",
" .attr("y", function(d) { return yScale(d.value); })\n",
" .attr("width", xSubgroup.bandwidth())\n",
" .attr("height", function(d) { return (CHART_HEIGHT - yScale(d.value)); })\n",
" .attr("fill", function(d) { return color(d.key); })\n",
" .style("opacity", "0.8");\n",
"\n",
"\n",
" return svgEl._groups[0][0].outerHTML;\n",
" }\n",
"\n",
"\n",
" const profileFromCSVfile = {"driver_rate_1m": {"frequentItems": [{"value": "3.000000", "estimate": 45}, {"value": "2.000000", "estimate": 34}, {"value": "1.000000", "estimate": 10}, {"value": "4.000000", "estimate": 9}, {"value": "5.000000", "estimate": 0}, {"value": "6.000000", "estimate": 0}, {"value": "7.000000", "estimate": 0}, {"value": "9.000000", "estimate": 0}, {"value": "10.000000", "estimate": 0}]}}\n",
"\n",
" Handlebars.registerHelper("getDoubleHistogramChart",(column,key) => {\n",
" const columnKey = key.data.key\n",
" try {\n",
" if (profileFromCSVfile) {\n",
" return generateBarChart(\n",
" column,\n",
" profileFromCSVfile[columnKey]\n",
" )\n",
" }\n",
" } catch (err) {\n",
" $(document).ready(() =>\n",
" $(".desktop-content").html(`\n",
" <p style="height: ${$(window).height()}px" class="error-message">\n",
" Something went wrong. Please try again.\n",
" </p>\n",
" `)\n",
" )\n",
" }\n",
" });\n",
" }\n",
"\n",
" function initWebsiteScripts() {\n",
" $(".svg-container").css("height", $(window).height() - 32)\n",
" }\n",
"\n",
" function initHandlebarsTemplate() {\n",
" // Replace this context with JSON from .py file\n",
" const context = {"driver_rate_1m": {"frequentItems": [{"value": "3.000000", "estimate": 22}, {"value": "1.000000", "estimate": 16}, {"value": "5.000000", "estimate": 15}, {"value": "4.000000", "estimate": 11}, {"value": "6.000000", "estimate": 8}, {"value": "2.000000", "estimate": 4}, {"value": "7.000000", "estimate": 3}, {"value": "9.000000", "estimate": 2}, {"value": "10.000000", "estimate": 1}]}};\n",
" // Config handlebars and pass data to HBS template\n",
" const source = document.getElementById("entry-template").innerHTML;\n",
" const template = Handlebars.compile(source);\n",
" const html = template(context);\n",
" const target = document.getElementById("generated-html");\n",
" target.innerHTML = html;\n",
" }\n",
"\n",
" // Invoke functions -- keep in mind invokation order\n",
" registerHandlebarHelperFunctions();\n",
" initHandlebarsTemplate();\n",
" initWebsiteScripts();\n",
" </script>\n",
"</html>\n",
"\" width=100% height=277px\n",
" frameBorder=0></iframe>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"visualization.distribution_chart(feature_name=\"driver_rate_1m\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There's a lot of ratings that don't even show in the reference profile, so it really likes like there's a significant drift here."
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div></div><iframe srcdoc=\"<!DOCTYPE html>\n",
"<html lang="en">\n",
" <head>\n",
" <meta charset="UTF-8" />\n",
" <meta http-equiv="X-UA-Compatible" content="IE=edge" />\n",
" <meta name="viewport" content="width=device-width, initial-scale=1.0" />\n",
" <meta name="description" content="" />\n",
" <meta name="author" content="" />\n",
"\n",
" <title>Profile Visualizer | whylogs</title>\n",
"\n",
" <link rel="icon" href="images/whylabs-favicon.png" type="image/png" sizes="16x16" />\n",
" <link rel="preconnect" href="https://fonts.googleapis.com" />\n",
" <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />\n",
" <link href="https://fonts.googleapis.com/css2?family=Asap:wght@400;500;600;700&display=swap" rel="stylesheet" />\n",
" <link rel="preconnect" href="https://fonts.gstatic.com" />\n",
" <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.0/dist/css/bootstrap.min.css" />\n",
"\n",
" <script\n",
" src="https://cdnjs.cloudflare.com/ajax/libs/handlebars.js/4.7.7/handlebars.min.js"\n",
" integrity="sha512-RNLkV3d+aLtfcpEyFG8jRbnWHxUqVZozacROI4J2F1sTaDqo1dPQYs01OMi1t1w9Y2FdbSCDSQ2ZVdAC8bzgAg=="\n",
" crossorigin="anonymous"\n",
" referrerpolicy="no-referrer"\n",
" ></script>\n",
"\n",
" <style type="text/css">\n",
"\n",
" :root {\n",
" /** Branded colors */\n",
" --brandSecondary900: #4f595b;\n",
" --secondaryLight1000: #313b3d;\n",
" /** Purpose colors */\n",
" --tealBackground: #eaf2f3;\n",
" }\n",
"\n",
" /* RESET STYLE */\n",
" *,\n",
" *::after,\n",
" *::before {\n",
" margin: 0;\n",
" padding: 0;\n",
" box-sizing: border-box;\n",
" overflow-y: hidden;\n",
" }\n",
"\n",
" /* Screen on smaller screens */\n",
" .no-responsive {\n",
" display: none;\n",
" position: fixed;\n",
" top: 0;\n",
" left: 0;\n",
" z-index: 1031;\n",
" width: 100vw;\n",
" height: 100vh;\n",
" background-color: var(--tealBackground);\n",
" display: flex;\n",
" align-items: center;\n",
" justify-content: center;\n",
" }\n",
"\n",
" .desktop-content {\n",
" display: flex;\n",
" flex-direction: column;\n",
" }\n",
"\n",
" .no-responsive__content {\n",
" max-width: 600px;\n",
" width: 100%;\n",
" padding: 0 24px;\n",
" }\n",
"\n",
" .no-responsive__title {\n",
" font-size: 96px;\n",
" font-weight: 300;\n",
" color: var(--brandSecondary900);\n",
" line-height: 1.167;\n",
" }\n",
"\n",
" .no-responsive__text {\n",
" margin: 0;\n",
" font-size: 16px;\n",
" font-weight: 400;\n",
" color: var(--brandSecondary900);\n",
" line-height: 1.5;\n",
" }\n",
"\n",
" .svg-container {\n",
" display: inline-block;\n",
" position: relative;\n",
" width: 100%;\n",
" vertical-align: top;\n",
" overflow: hidden;\n",
" }\n",
"\n",
" .svg-content-responsive {\n",
" display: inline-block;\n",
" position: absolute;\n",
" left: 0;\n",
" }\n",
"\n",
" .circle-color {\n",
" display: inline-block;\n",
" padding: 5px;\n",
" border-radius: 50px;\n",
" }\n",
"\n",
" .colors-for-distingushing-charts {\n",
" padding-right: 10px;\n",
" }\n",
"\n",
" .display-flex{\n",
" display: flex;\n",
" }\n",
"\n",
" .align-items-flex-end {\n",
" align-items: flex-end;\n",
" }\n",
"\n",
" .chart-box-title {\n",
" width: 88%;\n",
" justify-content: space-between;\n",
" margin: 10px;\n",
" margin-top: 15px;\n",
" bottom: 0;\n",
" }\n",
"\n",
" .chart-box-title p{\n",
" margin-bottom: 0;\n",
" font-family: Asap;\n",
" font-weight: bold;\n",
" font-size: 18px;\n",
" line-height: 16px;\n",
" color: #4F595B;\n",
" }\n",
"\n",
" .bar {\n",
" font: 10px sans-serif;\n",
" }\n",
"\n",
" .bar path,\n",
" .bar line {\n",
" fill: none;\n",
" stroke: #000;\n",
" shape-rendering: crispEdges;\n",
" }\n",
"\n",
" .error-message {\n",
" display: flex;\n",
" justify-content: center;\n",
" align-items: center;\n",
" color: rgb(255, 114, 71);\n",
" font-size: 30px;\n",
" font-weight: 900;\n",
" }\n",
"\n",
" @media screen and (min-width: 500px) {\n",
" .desktop-content {\n",
" display: block;\n",
" }\n",
" .no-responsive {\n",
" display: none;\n",
" }\n",
" }\n",
" </style>\n",
" </head>\n",
"\n",
" <body id="generated-html"></body>\n",
" <script id="entry-template" type="text/x-handlebars-template">\n",
" \n",
" <div class="desktop-content">\n",
"{{#each this}} <div class="chart-box" id="chart-box">\n",
" <div class="chart-box-title display-flex">\n",
" <p>{{@key}}</p>\n",
" <div class="display-flex">\n",
" <div class="colors-for-distingushing-charts">\n",
" <div class="circle-color" style="background: #44C0E7;"></div>\n",
" <text alignment-baseline="middle" style="font-size: 15px;">Target</text>\n",
" </div>\n",
" <div class="colors-for-distingushing-charts">\n",
" <div class="circle-color" style="background: #F5843C"></div>\n",
" <text alignment-baseline="middle" style="font-size: 15px;">Reference</text>\n",
" </div>\n",
" </div></div>\n",
" <div class="svg-container">{{{getDoubleHistogramChart this}}}</div>\n",
" </div>\n",
"{{/each}} </div>\n",
" <div class="no-responsive">\n",
" <div class="no-responsive__content">\n",
" <h1 class="no-responsive__title">Hold on! :)</h1>\n",
" <p class="no-responsive__text">\n",
" It looks like your current screen size or device is not yet supported by the WhyLabs Sandbox. The Sandbox is\n",
" best experienced on a desktop computer. Please try maximizing this window or switching to another device. We\n",
" are working on adding support for a larger variety of devices.\n",
" </p>\n",
" </div>\n",
" </div>\n",
" \n",
" </script>\n",
"\n",
" <script src="https://code.jquery.com/jquery-3.6.0.min.js" integrity="sha256-/xUj+3OJU5yExlq6GSYGSHk7tPXikynS7ogEvDej/m4=" crossorigin="anonymous"></script>\n",
"\n",
" <script src="https://cdnjs.cloudflare.com/ajax/libs/d3/6.7.0/d3.min.js" integrity="sha512-cd6CHE+XWDQ33ElJqsi0MdNte3S+bQY819f7p3NUHgwQQLXSKjE4cPZTeGNI+vaxZynk1wVU3hoHmow3m089wA==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>\n",
"\n",
" <script>\n",
" function registerHandlebarHelperFunctions() {\n",
" //helper fun\n",
"\n",
" class GenerateChartParams {\n",
" constructor(height, width, targetData, referenceData, bottomMargin=30, topMargin=5) {\n",
" this.MARGIN = {\n",
" TOP: topMargin,\n",
" RIGHT: 5,\n",
" BOTTOM: bottomMargin,\n",
" LEFT: 55,\n",
" };\n",
" this.SVG_WIDTH = width;\n",
" this.SVG_HEIGHT = height;\n",
" this.CHART_WIDTH = this.SVG_WIDTH - this.MARGIN.LEFT - this.MARGIN.RIGHT;\n",
" this.CHART_HEIGHT = this.SVG_HEIGHT - this.MARGIN.TOP - this.MARGIN.BOTTOM;\n",
" this.svgEl = d3.create("svg")\n",
" .attr("preserveAspectRatio", "xMinYMin meet")\n",
" .attr("viewBox", `0 0 ${$(window).width()+100} ${$(window).height()-30}`)\n",
" .classed("svg-content-responsive", true)\n",
" this.maxYValue = d3.max(targetData, (d) => Math.abs(d.axisY));\n",
" this.minYValue = d3.min(targetData, (d) => Math.abs(d.axisY));\n",
" const mergedReferenceData = referenceData.map(({axisX, axisY}) => {\n",
" return {axisX, axisY}\n",
" })\n",
" const mergedTargetedData = targetData.map(({axisX, axisY}) => {\n",
" return {axisX, axisY}\n",
" })\n",
"\n",
" this.charts2 = mergedReferenceData.concat(mergedTargetedData)\n",
" this.charts2 = this.charts2.sort(function(a, b) { return a - b; });\n",
"\n",
" this.targetBinWidth = targetData[1]?.axisX - targetData[0]?.axisX\n",
" this.referenceBinWidth = referenceData[1]?.axisX - referenceData[0]?.axisX\n",
" this.maxTargetXValue = d3.max(targetData, (d) => d.axisX);\n",
"\n",
" this.maxReferenceXValue = d3.max(referenceData, (d) => d.axisX);\n",
"\n",
" this.xScale = d3\n",
" .scaleLinear()\n",
" .domain([d3.min(this.charts2, function(d) { return parseFloat(d.axisX); }),(this.maxTargetXValue+this.targetBinWidth >= this.maxReferenceXValue+this.referenceBinWidth) ? this.maxTargetXValue+this.targetBinWidth:this.maxReferenceXValue+this.referenceBinWidth])\n",
" .range([0, this.CHART_WIDTH ]);\n",
"\n",
" this.svgEl.append("g")\n",
" .attr("transform", "translate("+ this.MARGIN.LEFT +"," + this.SVG_HEIGHT + ")")\n",
" .call(d3.axisBottom(this.xScale));\n",
" this.yScale = d3.scaleLinear()\n",
" .range([this.CHART_HEIGHT , 0])\n",
" .domain([d3.min(this.charts2, function(d) { return parseFloat(d.axisY); }), d3.max(this.charts2, function(d) { return parseFloat(d.axisY); })*1.2])\n",
" .nice();\n",
" }\n",
" }\n",
"\n",
" function chartData(column) {\n",
" const data = [];\n",
" if (column?.histogram) {\n",
" for (let i = 0; i<column.histogram.bins.length; i++) {\n",
" data.push({\n",
" axisY: column.histogram.counts[i] || 0,\n",
" axisX: column.histogram.bins[i] || 0,\n",
" });\n",
" }\n",
" } else {\n",
" $(document).ready(() =>\n",
" $(".desktop-content").html(`\n",
" <p style="height: ${$(window).height()}px" class="error-message">\n",
" Something went wrong. Please try again.\n",
" </p>\n",
" `)\n",
" )\n",
" }\n",
"\n",
" return data\n",
" }\n",
"\n",
" function generateDoubleHistogramChart(targetData, referenceData) {\n",
" let histogramData = [],\n",
" overlappedHistogramData = [];\n",
" let yFormat;\n",
"\n",
" histogramData = chartData(targetData)\n",
" overlappedHistogramData = chartData(referenceData)\n",
"\n",
" const sizes = new GenerateChartParams($(window).height()-80, $(window).width(), histogramData, overlappedHistogramData)\n",
" const {\n",
" MARGIN,\n",
" SVG_WIDTH,\n",
" SVG_HEIGHT,\n",
" CHART_WIDTH,\n",
" CHART_HEIGHT,\n",
" svgEl,\n",
" maxYValue,\n",
" minYValue,\n",
" xScale,\n",
" yScale\n",
" } = sizes\n",
"\n",
" const rectColors = ["#44C0E7", "#F5843C"]\n",
" const yAxis = d3.axisLeft(yScale).ticks(SVG_HEIGHT / 40);\n",
"\n",
" svgEl.append("g")\n",
" .attr("transform", `translate(${MARGIN.LEFT}, ${MARGIN.BOTTOM})`)\n",
" .call(yAxis)\n",
" .call(g => g.select(".domain").remove())\n",
" .call(g => g.selectAll(".tick line")\n",
" .attr("x2", CHART_WIDTH)\n",
" .attr("stroke-opacity", 0.1))\n",
" .call(g => g.append("text")\n",
" .attr("x", -MARGIN.LEFT)\n",
" .attr("y", 10)\n",
" .attr("fill", "currentColor")\n",
" .attr("text-anchor", "start"))\n",
"\n",
" svgEl.append("text")\n",
" .attr("transform",\n",
" "translate(" + (CHART_WIDTH/2) + " ," +\n",
" (CHART_HEIGHT + MARGIN.TOP + 75) + ")")\n",
" .style("text-anchor", "middle")\n",
" .text("Values")\n",
" .style("font-size", "15")\n",
" .style("opacity", "0.6")\n",
"\n",
" svgEl.append("text")\n",
" .attr("transform", "rotate(-90)")\n",
" .attr("y", 0)\n",
" .attr("x", 0 - (SVG_HEIGHT / 2))\n",
" .attr("dy", "1em")\n",
" .style("text-anchor", "middle")\n",
" .text("Counts")\n",
" .style("font-size", "15")\n",
" .style("opacity", "0.6")\n",
"\n",
" const gChart = svgEl.append("g");\n",
" gChart\n",
" .attr("transform", "translate("+ MARGIN.LEFT +",0)")\n",
" .selectAll(".bar")\n",
" .data(histogramData)\n",
" .enter()\n",
" .append("rect")\n",
" .style("stroke", "#021826")\n",
" .classed("bar", true)\n",
" .attr("width", function(d) { return xScale(histogramData[1]?.axisX)-xScale(histogramData[0]?.axisX); })\n",
" .attr("height", (d) => CHART_HEIGHT - yScale(d.axisY))\n",
" .attr("x", 1)\n",
" .attr("transform", function(d) { return "translate(" + xScale(d.axisX) + "," + 0 + ")"; })\n",
" .attr("y", (d) => yScale(d.axisY) + MARGIN.TOP + MARGIN.BOTTOM)\n",
" .attr("fill", rectColors[0])\n",
" .style("opacity","0.6")\n",
"\n",
" const gChart1 = svgEl.append("g");\n",
" gChart1\n",
" .attr("transform", "translate("+ MARGIN.LEFT +",0)")\n",
" .selectAll(".bar")\n",
" .data(overlappedHistogramData)\n",
" .enter()\n",
" .append("rect")\n",
" .style("stroke", "#021826")\n",
" .classed("bar", true)\n",
" .attr("width", function(d) { return xScale(overlappedHistogramData[1]?.axisX)-xScale(overlappedHistogramData[0]?.axisX); })\n",
" .attr("height", (d) => CHART_HEIGHT - yScale(d.axisY))\n",
" .attr("x", 1)\n",
" .attr("transform", function(d) { return "translate(" + xScale(d.axisX) + "," + 0 + ")"; })\n",
" .attr("y", (d) => yScale(d.axisY) + MARGIN.TOP + MARGIN.BOTTOM)\n",
" .attr("fill", rectColors[1])\n",
" .style("opacity","0.6")\n",
"\n",
" return svgEl._groups[0][0].outerHTML;\n",
" }\n",
"\n",
" const profileFromCSVfile = {"driver_avg_daily_trips": {"histogram": {"start": 9.0, "end": 44.0000044, "width": 0, "counts": [5, 0, 0, 0, 1, 0, 2, 4, 0, 3, 4, 4, 22, 0, 8, 0, 0, 9, 9, 18, 6, 0, 0, 0, 3], "max": 44.0, "min": 9.0, "bins": [9.0, 10.400000176, 11.800000352, 13.200000528, 14.600000704, 16.00000088, 17.400001056, 18.800001232, 20.200001408, 21.600001584, 23.00000176, 24.400001936000002, 25.800002112, 27.200002288, 28.600002464, 30.00000264, 31.400002816, 32.800002992, 34.200003168, 35.600003344, 37.00000352, 38.400003696, 39.800003872000005, 41.200004048000004, 42.600004224, 44.0000044], "n": 98}}}\n",
"\n",
" Handlebars.registerHelper("getDoubleHistogramChart",(column,key) => {\n",
" const columnKey = key.data.key\n",
" try {\n",
" if (profileFromCSVfile) {\n",
" return generateDoubleHistogramChart (\n",
" column,\n",
" profileFromCSVfile[columnKey]\n",
" )\n",
" }\n",
" } catch (err) {\n",
" $(document).ready(() =>\n",
" $(".desktop-content").html(`\n",
" <p style="height: ${$(window).height()}px" class="error-message">\n",
" Something went wrong. Please try again.\n",
" </p>\n",
" `)\n",
" )\n",
" }\n",
" });\n",
" }\n",
"\n",
" function initWebsiteScripts() {\n",
" $(".svg-container").css("height", $(window).height() - 32)\n",
" }\n",
"\n",
" function initHandlebarsTemplate() {\n",
" // Replace this context with JSON from .py file\n",
" const context = {"driver_avg_daily_trips": {"histogram": {"start": 0.0, "end": 20.000002, "width": 0, "counts": [20, 5, 1, 12, 7, 9, 7, 2, 3, 0, 0, 0, 4, 4, 0, 0, 2, 0, 5, 0, 1], "max": 20.0, "min": 0.0, "bins": [0.0, 0.9523810476190475, 1.904762095238095, 2.8571431428571428, 3.80952419047619, 4.761905238095237, 5.7142862857142855, 6.666667333333333, 7.61904838095238, 8.571429428571427, 9.523810476190475, 10.476191523809522, 11.428572571428571, 12.380953619047618, 13.333334666666666, 14.285715714285713, 15.23809676190476, 16.19047780952381, 17.142858857142855, 18.095239904761904, 19.04762095238095, 20.000002], "n": 82}}};\n",
" // Config handlebars and pass data to HBS template\n",
" const source = document.getElementById("entry-template").innerHTML;\n",
" const template = Handlebars.compile(source);\n",
" const html = template(context);\n",
" const target = document.getElementById("generated-html");\n",
" target.innerHTML = html;\n",
" }\n",
"\n",
" // Invoke functions -- keep in mind invokation order\n",
" registerHandlebarHelperFunctions();\n",
" initHandlebarsTemplate();\n",
" initWebsiteScripts();\n",
" </script>\n",
"</html>\n",
"\" width=100% height=300px\n",
" frameBorder=0></iframe>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"visualization.double_histogram(feature_name=\"driver_avg_daily_trips\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Likewise, these histograms almost don't overlap, so it's pretty clear that these distributions are different."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `NotebookProfileVisualizer` has a bunch of other types of features and visualization. If you like to know more, be sure to check the example on the [Notebook Profile Visualizer](https://whylogs.readthedocs.io/en/stable/examples/basic/Notebook_Profile_Visualizer.html)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Appendix - Changing the Dataset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This section is not really a part of the demonstration. It's just to show the changes made in the dataset that originated the `driver_stats_changed.parquet` file that will be used in the beginning of the notebook."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Driver Statistics\n",
"\n",
"The NYC taxi datasets provides only information about rides, but in this example we want to show an example of using an online feature store to enrich ride information with driver statistics. So, we'll fabricate some driver statistics and link them with the rides dataset (__nyc_taxi_rides_feb_2020.parquet__) through the `Driver_ID` key."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"\n",
"\n",
"dstats = pd.DataFrame(\n",
" {'event_timestamp': pd.date_range('2020-02-10', '2020-02-17', freq='1H', closed='left')}\n",
" )\n",
"dstats['driver_id'] = '1001'\n",
"\n",
"dstats2 = pd.DataFrame(\n",
" {'event_timestamp': pd.date_range('2020-02-10', '2020-02-17', freq='1H', closed='left')}\n",
" )\n",
"dstats2['driver_id'] = '1002'\n",
"\n",
"dstats_tot = pd.concat([dstats, dstats2])"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"dstats_tot = dstats_tot.sort_values(by=[\"event_timestamp\",\"driver_id\"])"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [],
"source": [
"import datetime\n",
"dstats_tot['created'] = datetime.datetime.now()"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"mu, sigma = 30, 6 # mean and standard deviation\n",
"s = np.random.normal(mu, sigma, len(dstats_tot))\n",
"daily_trips = np.round(s)\n",
"daily_trips = [int(x) for x in daily_trips]\n",
"dstats_tot['avg_daily_trips'] = daily_trips"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [],
"source": [
"from scipy.stats import truncnorm\n",
"\n",
"def get_truncated_normal(mean=3, sd=0.75, low=1, upp=11):\n",
" return truncnorm(\n",
" (low - mean) / sd, (upp - mean) / sd, loc=mean, scale=sd)\n",
"\n",
"X = get_truncated_normal()\n",
"\n",
"dstats_tot['rate_1m'] = [int(x) for x in X.rvs(len(dstats_tot))]"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"mu, sigma = 20, 4 # mean and standard deviation\n",
"s = np.random.normal(mu, sigma, len(dstats_tot))\n",
"avg_speed = np.round(s,2)\n",
"avg_speed\n",
"dstats_tot['avg_speed'] = avg_speed"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Adding changes - Stats Update Frequency"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [],
"source": [
"dstats_tot = dstats_tot.reset_index()\n",
"cond = (dstats_tot['event_timestamp'].dt.day==11) & (dstats_tot['event_timestamp'].dt.month==2) & ((dstats_tot['event_timestamp'].dt.hour%2)!=0)\n",
"df2 = dstats_tot.loc[cond]\n",
"dstats_tot = dstats_tot[~dstats_tot.isin(df2)].dropna()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Adding changes - Rate_1m\n",
"\n",
"We're assuming that this change in customer's behaviour would not change the mean of the distribution, but would have an increased standard deviation, making the rates be more spreaded, increasing the frequency of extreme ratings (positive or negative)."
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"cond = (dstats_tot['event_timestamp'].dt.day==14) & (dstats_tot['event_timestamp'].dt.month==2)\n",
"size = len(dstats_tot.loc[cond])\n",
"\n",
"X = get_truncated_normal(mean=3, sd=2, low=1, upp=11)\n",
"rate_1m = [int(x) for x in X.rvs(size)]\n",
"dstats_tot.loc[cond, 'rate_1m'] = rate_1m\n",
"\n",
"import numpy as np\n",
"cond = (dstats_tot['event_timestamp'].dt.day==15) & (dstats_tot['event_timestamp'].dt.month==2)\n",
"size = len(dstats_tot.loc[cond])\n",
"\n",
"X = get_truncated_normal(mean=3, sd=3, low=1, upp=11)\n",
"rate_1m = [int(x) for x in X.rvs(size)]\n",
"dstats_tot.loc[cond, 'rate_1m'] = rate_1m\n",
"\n",
"import numpy as np\n",
"cond = (dstats_tot['event_timestamp'].dt.day==16) & (dstats_tot['event_timestamp'].dt.month==2)\n",
"size = len(dstats_tot.loc[cond])\n",
"\n",
"X = get_truncated_normal(mean=3, sd=4, low=1, upp=11)\n",
"rate_1m = [int(x) for x in X.rvs(size)]\n",
"dstats_tot.loc[cond, 'rate_1m'] = rate_1m"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Adding Changes - Avg Daily Trips"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {},
"outputs": [],
"source": [
"cond = (dstats_tot['event_timestamp'].dt.day==14) & (dstats_tot['event_timestamp'].dt.month==2)\n",
"size = len(dstats_tot.loc[cond])\n",
"\n",
"mu, sigma = 24, 6 # mean and standard deviation\n",
"s = np.random.normal(mu, sigma, size)\n",
"daily_trips = np.round(s)\n",
"daily_trips = [int(x) if x>0 else 0 for x in daily_trips]\n",
"# daily_trips\n",
"\n",
"dstats_tot.loc[cond, 'avg_daily_trips'] = daily_trips\n",
"\n",
"cond = (dstats_tot['event_timestamp'].dt.day==15) & (dstats_tot['event_timestamp'].dt.month==2)\n",
"size = len(dstats_tot.loc[cond])\n",
"\n",
"mu, sigma = 12, 6 # mean and standard deviation\n",
"s = np.random.normal(mu, sigma, size)\n",
"daily_trips = np.round(s)\n",
"daily_trips = [int(x) if x>0 else 0 for x in daily_trips]\n",
"# daily_trips\n",
"\n",
"dstats_tot.loc[cond, 'avg_daily_trips'] = daily_trips\n",
"\n",
"cond = (dstats_tot['event_timestamp'].dt.day==16) & (dstats_tot['event_timestamp'].dt.month==2)\n",
"size = len(dstats_tot.loc[cond])\n",
"\n",
"mu, sigma = 3, 6 # mean and standard deviation\n",
"s = np.random.normal(mu, sigma, size)\n",
"daily_trips = np.round(s)\n",
"daily_trips = [int(x) if x>0 else 0 for x in daily_trips]\n",
"# daily_trips\n",
"\n",
"dstats_tot.loc[cond, 'avg_daily_trips'] = daily_trips\n",
"\n",
"dstats_tot = dstats_tot.astype({'driver_id': 'int64','avg_daily_trips':'int64','rate_1m':'int64'})\n",
"# dstats_tot.to_parquet(\"driver_stats.parquet\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Rides Dataset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `nyc_taxi_rides_feb_2020.parquet` was extracted from [the TLC trip record data](https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page). We randomly sampled the data and selected a few chosen features, in order to reduce the dataset for this demonstration.\n",
"\n",
"In addition, one features was created: The day of the week, based from `tpep_pickup_datetime`.\n",
"\n",
"The original features are described in this [data dictionary](https://www1.nyc.gov/assets/tlc/downloads/pdf/data_dictionary_trip_records_yellow.pdf)."
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "Feast_v1.ipynb",
"provenance": []
},
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.10"
},
"vscode": {
"interpreter": {
"hash": "5dd5901cadfd4b29c2aaf95ecd29c0c3b10829ad94dcfe59437dbee391154aea"
}
}
},
"nbformat": 4,
"nbformat_minor": 0
}