examples/quantile_regression.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Using quantile regression as uncertainty surrogate"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Usually, Bayesian models such as Gaussian processes are used to determine uncertainty intervalls. But there are many other techniques that can be used as uncertainty surrogates. For example, PyePAL implements [quantile regression](https://en.wikipedia.org/wiki/Quantile_regression) using [Gradient Boosted Decision trees](https://en.wikipedia.org/wiki/Gradient_boosting)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As in the other examples, we will use the [Binh-Korn test function](https://en.wikipedia.org/wiki/Test_functions_for_optimization)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from pyepal.models.gbdt import build_gbdt_tuple\n",
"from pyepal import PALGBDT\n",
"from pyepal.pal.utils import exhaust_loop\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt \n",
"plt.style.use('ggplot')"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"def binh_korn(x, y): # pylint:disable=invalid-name\n",
" \"\"\"https://en.wikipedia.org/wiki/Test_functions_for_optimization\"\"\"\n",
" obj1 = 4 * x ** 2 + 4 * y ** 2\n",
" obj2 = (x - 5) ** 2 + (y - 5) ** 2\n",
" return -obj1, -obj2\n",
"\n",
"def binh_korn_points():\n",
" \"\"\"Create a dataset based on the Binh-Korn test function\"\"\"\n",
" x = np.linspace(0, 5, 100) # pylint:disable=invalid-name\n",
" y = np.linspace(0, 3, 100) # pylint:disable=invalid-name\n",
" array = np.array([binh_korn(xi, yi) for xi, yi in zip(x, y)])\n",
" return np.hstack([x.reshape(-1, 1), y.reshape(-1, 1)]), array"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"x, points = binh_korn_points()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Again, we can start by plotting our objective space."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Text(0, 0.5, 'objective 2')"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEJCAYAAACKWmBmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAvl0lEQVR4nO3deXxU5aHG8d97sgBhzWQFWWMSQZHNBCEsVYnWi9ZaWyygVhFKKyi3RSxYQalRgYp65RK9WhCQekW0UpfriqgosezIJkukIksgJJMQiEAI894/RkciSQzLZGaS5/v55ENyZuacZ45OnpztPcZaaxEREamCE+gAIiIS3FQUIiJSLRWFiIhUS0UhIiLVUlGIiEi1VBQiIlKt8EAH8Ie9e/eeMi02NpaCgoIApDlzylw7lNn/Qi0v1L/MrVq1qvIxbVGIiEi1VBQiIlItFYWIiFRLRSEiItVSUYiISLVUFCIiUi0VhYiIVKtOXkdxJuzer7ErP4FmLTBNm0OzFtC0hfffqMYYYwKcUEQkMFQU37J7vsb+30KwllNu0BEW7i2MZi2geTSmeXTF75u7oIULmkVjIiJqPbuIiD+pKL7lpPfF9ugNpSVQUgwlB7ElxXDo4Lc/F2NLiqCoALszF0oOgvWcWipNmvqKw7SIgegYaBGDaeHyfh8dC02aaQtFREKGiuIkJiwMmkV7v4DqfpVbzwk4XALFRXCwCFtcCAeL4KAbW+yGokLs7p3ekvlhoYRHeEvDFYf5rjxccZiYeHDFQUwspmGUH9+piEjNqSjOkHFqVir2xAlvgRQXQnEhtqgQ3AXeLZOiQuz2zd7HTpyoWCZRjSmMb8mJ5i5MbALExHuLJDYeYhIwjZv4+y2KiAAqCr8zYWHgivV+UXmhWM8JOFgM7gNY9wFwH4DCAziHiiFvN3bLBjh25JQiITYBYhO9RRKXgIlrCXGJ3q2TcP2nFZFzQ79NgoBxwr49fhGDOb+jb3r0tyNBWmvhm8NQkA+F+7EF+6FgP/bAfti7E7t+JZQf/75IHAdi4iEuERPfEuJbYeJbQUJLiE3AhOuAu4jUnIoiBBhjoHFT71e780/ZKrEeDxS7oWAfNj8PDuyDA97v7fKlcKT01BJJOA+TeB4ktMIknAeJrb0H4HWQXUR+QEVRBxjH8e3eMqmdKzxmrYXDhyB/L3b/XsjfC/l52H27sds2Qtmx70ukQSNIPA/TsjUktsa0bAMt23i3TLQrS6Te0qe/jjPGQNNm0LRZhd1a8G2JFBXC/j3YfXtg325vgWzdCP/66PsCCQv3bnm0agut2mJatYFWbb27tMLCav09iUjtUlHUY8aY77dEOnWt8Jg9+g3k7cHm7YK8Xdi8Xd7rR1Yv8xYMQHg4JLbBtG4HrdphzmvLiS6XYK3RLiyROkRFIZUyDaOgQwqmQ0qF6fbYMe+Wx96vYc9X3ivaT9oCKQDvsZTW7TGt28N57TBtk7xFoqvWRUKSikJOi2nQwHtAvd35Fabbbw7D7p00Li7g8NaN2F3/xn7y3vfHQMLCvMc92iZBmyRM2/OhbRKmkS4sFAl2Kgo5J0xUE0i9iKjYWL7p+RPg27OxDuyDXTu8xfH1Duzmz+GzD78//hHfyls67ZK//1flIRJUVBTiN8ZxIKGV90B4Wl/fdFtSBDt3YHfmYr/+EvvlFlj5ibc8jPGeuts+GdqnYNqneLc8IiID9j5E6jsVhdQ60ywaLr4Ec/Elvmn2UAns3I79Khf71XbsF+u/P/MqLNx7zCMpFTpcgEm6AOJb6oC5SC1RUUhQME2bQedLMJ1PKo+iQvj3Nux3XzkfwodvecujSTNI8paGSe7k3fpo0DBg+UXqMhWFBC3vyLq9MT16A9+OibV3F3bHVtixBfvlVuz6ld7icBzvQfKUC73FcX4n79DuInLWVBQSMowT9v1pt/1/CoAtPQQ7tmJzt2BzN2OXvoNd/Lr3BXGJmNSLIKWz99/YBO2uEjkDKgoJaaZxU7g4DXNxGgC2/Djs+jd2+2bv1+crYNkH3q2OFjHeIU4u6Iy54GId5xCpIRWF1CkmPAI6pGI6pMJV13tP0d23G7ttE2zbiN26HlZ8/G1xuDCpF3MkPQPbOsk7XLuInEJFIXWacZxvx6dqC5f9h3f4kf17sVs3wNYN2C2fU7LiY++T4xIxHbtAp66Yjl29B9hFREUh9YsxxjtCbuJ58JOrsdYSfeQQ7s+WYrd8jl21DD55D2uM9+D4hd0wF3aD5As1BInUWyoKqdeMMYS3TcKJagYDrvXeunZnLnbzOuwX67Dv/xP7zj8gsgF07ILp3ANzUQ/vDaFE6omAF8Vnn33Gyy+/zJ49e3jkkUc4//zvxxBatGgRS5YswXEchg0bRrdu3QIXVOoFExbmuz6Da3/tHUV360bspjXYjWu+Px03viXm4jTvdR8XdNaV41KnBbwo2rRpw7hx43j22WcrTN+9ezc5OTk8/vjjFBUVkZWVxZNPPonjOAFKKvWRaRgFXXtiuvYEwObvxW5ai92wGrv0XewHb3i3Njp1xXRJw3RJx7SICXBqkXMr4EXRunXrSqevXLmSjIwMIiIiiI+PJzExkdzcXFJTU2s5ocj3zHf3H7/8GmzZMe8B8Q2rsOtXYT9f4d3aaJfsLYxuPb3HOXQKroS4gBdFVdxuNykp398LweVy4Xa7K33u4sWLWbx4MQBTp04lNjb2lOeEh4dXOj2YKXPtOKvMrc6Dy70HxU98vYNjq5ZxbOWnHH9zAfaNF3HiEmlwaX8aXNqfiE5dMGHn5iMXaus51PKCMleY7zmfYyWysrIoLi4+ZfrgwYNJT08/6/lnZmaSmZnp+7mgoOCU58TGxlY6PZgpc+04Z5kbN4efDISfDMQpKcauX4ln3XK+eWcR37y5EBo3xXTriemRAZ26ndVZVKG2nkMtL9S/zK1atarysVopikmTJp32a1wuF4WFhb6f3W43LpfG7pHQYJq1wPS9EvpeiT16BDavxa75zPu17ANoFIW5OB1zSQZ07oGJbBDoyCJVCtpdT2lpacyYMYNrr72WoqIi8vLySE5ODnQskdNmGjaCHhmYHhnY48dhy+fY1TnYz5djV3wMDRphuvbEpPeBi3roDCoJOgEvihUrVvDcc89RUlLC1KlTad++Pffddx9t2rShd+/ejB07FsdxGD58uM54kpBnIiJ8Y1PZEydg63rsqmXeLY0VH3u3NLr1wvTs7z2TKiws0JFFMNZa++NPCy179+49ZVp9298YKMp8Zmx5OWxZj131CXbNZ3DkG2jaHJPWB3PpZd5rO046eyoYMp+OUMsL9S9zwI9RiEj1THi491hF5x7Ym+6ADauxK5ZiP12M/fAt7zhUvS7zfsVX/YEW8QcVhUiQMRGR0MN7wyZ75Bvsmhzsvz7CvvkS9o0FkNyJb668DtuxGyaqcaDjSj2gohAJYqZRFKZPJvTJxLoLsMs/xn62hENPT4OISEz33pg+A7zjUOkYnviJikIkRBhXLOY/fom9+gZaFB2g6O1XsSs+9h4Ej4nH9MnE9BmAccUFOqrUMSoKkRBjjCEi9UIcVzz2xtu/vTZjMfb1/8W+8SJ0vgSn/1VwcbrOmpJzQkUhEsJMRCTm0p/ApT/BHtjnLYxli/FkP+K99WvfKzH9rsK4QmsoCgkuKgqROsLEJWKuvxn7syGwYSWej9/F/t9L2LcWQteeOJcN9F6boUEK5TSpKETqGBMWBt16EdatF7ZgP3bpO9hP3sez9l/eu/tddg0m4wpMo6hAR5UQodMkROowE5uAc8OtOH+dgxk+Fho1xi54Fs+fhuFZ8Dfs/lMvThX5IW1RiNQDJiIC0+sy6HUZ9t/bsEvexH70NnbJm9AlHSfzOrjgYu2WkkqpKETqGdMhFTN8LPZXw7xl8dFbeD5fAW06YK68HpPez3uluMi3tOtJpJ4yzaNxfj4UZ9pszG/uhPJy7HNP4LlvJJ73X/PeL1wEbVGI1HsmsgGm31XYPpmwcTWed1/FLpyNfXMB5rJrMJk/wzRtHuiYEkAqChEB8A4B0iWdsC7p2B1bvYXx9svYxf/E9Psp5qpf6HqMekpFISKnMEkXEHbHvdi8Xdi3/4H96C3sR297hwn5j19iYhMCHVFqkYpCRKpkWrbB3P4H7M+HYt95Ffvpe9hl72N6X4EZOAgTlxjoiFILVBQi8qNMTDzmpt9jBw7Cvvsqdum72M+WYDIGYK65ERMTH+iI4kcqChGpMRMdgxn8W+zVv8S+/Yr3qu+cJZh+V3oLo0VMoCOKH6goROS0mRYuzJCR2J/+AvvWy9hP3sPmfIC5/FrvMYzGTQMdUc4hXUchImfMuOJwbh6Fk/U0pkcG9r1FeO4dieetl7HHjgU6npwjKgoROWsmLhFn+Fic+5+E1Iuwi+bjmfh7PJ++j/WcCHQ8OUsqChE5Z0zr9oTdORHnnikQHYOd9994HvwDx9YtD3Q0OQsqChE550zqRTj3Porzuz9B2TGK//JHTjz5F2zerkBHkzOgohARvzDGYNL64vwlmya33glffoFn8l3e4c1LDwc6npwGnfUkIn5lIiJofP1Qvul6Kfa1F7BL/g+7/GPML27B9M3EOLqvd7DTFoWI1ArTtLn3DKmJj0PL1tj52XgeuQf7722BjiY/QkUhIrXKtE3CuWcKZsTdUOzGM+UePPOfwpYeCnQ0qYJ2PYlIrTPGYC79CbZLOvb1F7FL3sCuycH8ejjm0st0p70goy0KEQkY0ygK59fDcSY9AfEtsbOfwPPE/bqXd5BRUYhIwJnWHXDGT8UM/T18tR3PX8Z474dxQhfrBQMVhYgEBeOE4Vw+EOfBbLioB/aVuXim3IPd/e9AR6v3VBQiElRMixicUffi/H48uA/geWgsnjcXYMvLAx2t3lJRiEjQMcZgLumD82A25pK+2Nf+17t1sWdnoKPVSyoKEQlapkkznN/ejXPHBCgqwPPQH/G8twjr8QQ6Wr0S8NNj58+fz+rVqwkPDychIYFRo0bRuHFjABYtWsSSJUtwHIdhw4bRrVu3wIYVkYAwPTJwUi7C83w29uU52PWrcIb9ARMTF+ho9ULAtyi6dOnCY489xvTp02nZsiWLFi0CYPfu3eTk5PD4449z3333MXv2bDz6K0Kk3jJNm+OMuhdz2xj4Ktd7ZtTKTwIdq14IeFF07dqVsDDvWC+pqam43W4AVq5cSUZGBhEREcTHx5OYmEhubm4go4pIgBljcPpk4jzwpHcYkGcfxTN3BvbokUBHq9Oq3fVUXFzMU089xdatW2nTpg233HILF1xwge/xW2+9lXnz5p2zMEuWLCEjIwMAt9tNSkqK7zGXy+UrkR9avHgxixcvBmDq1KnExsae8pzw8PBKpwczZa4dyux/5zxvbCx22t8ofWk2pf94HuerbTQf9xAR7ZPP2SJCbR2D/zJXWxRz5swhOjqayZMns3nzZqZNm8btt99O3759AbDW1mghWVlZFBcXnzJ98ODBpKenA/Dqq68SFhZGv379TvMtQGZmJpmZmb6fCwoKTnlObGxspdODmTLXDmX2P7/l/ekvcdqlcGLWY7j/NAIzeASm30/PyRAgobaO4ewyt2rVqsrHqi2KzZs3k52dTWRkJB06dKBz585MmTKFo0ePkpmZWeP/GJMmTar28Y8++ojVq1dz//33++bpcrkoLCz0PcftduNyuWq0PBGpP0zHLjj3/xee2U9g5z8F2zbBLaMxDRoGOlqdUe0xihMnTlQ4gNyuXTsmT57MokWLePPNN89JgHXr1vHaa68xfvx4GjRo4JuelpZGTk4Ox48fJz8/n7y8PJKTz91mpYjUHaZZNM5/Tsb8fCh2xVLvNRf79gQ6Vp1R7RZFUlIS69ato1evXr5piYmJTJ48mQcffJCjR4+edYDZs2dTXl5OVlYWACkpKYwcOZI2bdrQu3dvxo4di+M4DB8+HMcJ+LF3EQlSxnEw1w7GdrgAz6zpeB65G+f2P2C69frxF0u1jK3mQMP27dspLS2t9PoFt9vNkiVL+NWvfuXPfGdk795TR56sb/sbA0WZa0eoZa7tvLbwAJ6np8DOXMzPhmCu/TXmNP/QDLV1DAE6RnHyWUc/5HK5grIkRERMTBzO+KnY+U9h33gRu2sHzvA/YhpGBTpaSNK+HBGpk0xEJGbYf2J+PQLWr8QzdTy2MD/QsUKSikJE6ixjDE7mdThjHgB3AZ6H78bmfhHoWCFHRSEidZ65qDvOvY9Coyg8j03Ervo00JFCymkVhcfjoaioyF9ZRET8xrRs7S2L9sl4nvkrnnf+UeOLhuu7GhVFaWkpTz75JDfddBNjxowBYNWqVSxYsMCv4UREziXTpBnO2CxMej/sP+ZhX3wG69HtVn9MjYrib3/7G1FRUTz11FOEh3tPlEpNTSUnJ8ev4UREzjUTEYkZcTfmql9gP3wLz7OPYo+XBTpWUKvR/Sg2bNjAM8884ysJgGbNmnHw4EG/BRMR8RfjOJhBw/A0j8a+/Byew4dwRt+HaaTTZytToy2KqKgoDh06VGFaQUEB0dHRfgklIlIbnKuuxwwfC9s3eQ9yHyoJdKSgVKOiGDBgAI899hgbN27EWsu2bdvIzs7myiuv9Hc+ERG/cnpdhjPqPtj7NZ5H78UWF/74i+qZGhXFz3/+czIyMpg9ezYnTpzg6aefJi0tjYEDB/o7n4iI35mu6Tj/Odl7rcVf78UWHgh0pKBSo2MUxhgGDhyoYhCROstc0Bnnj3/B8+Rf8Dx6L+UPZUN4gx9/YT1Qoy2Ke+65h9dff73C/SFEROoac35HnLuz4Mg3FE26E3tgX6AjBYUaFcWgQYPIzc3lD3/4Aw888ADvv/8+hw8f9nc2EZFaZ9ol49z9EPboN94D3AX7Ax0p4GpUFD179mTs2LE8++yzXH755axYsYI77riDadOm+TufiEitM22TiJ48A46U4pl+H9YdWsONn2unNYRHo0aN6Nu3L1dddRXJycmsXbvWX7lERAIq4vwLcP74IJQewvPEJGxJcaAjBUyNisJay4YNG3j66af57W9/y8svv0z37t2ZOXOmv/OJiASMaZ/y7cizB/A8cT+2tH7ucq/RWU+/+93vaNiwIRkZGWRlZdG6dWt/5xIRCQom5UKc0ffh+e8sPDOzcP7wIKZB/TobqkZF8ac//Ynk5GR/ZxERCUrmwu44I+72jjr7zDScUX/GhNfo12edUOWup/z87+8E1bRpU/bv31/pl4hIfWAu6YMZ+jvYsAr79+x6NUR5lZU4btw4nn/+eQDf0OKVeemll859KhGRIORcNhDPwSLsmy9BTALmZ4MDHalWVFkU35UEqAxERL5jrhsKhfnY1/8XT0wcTsaAQEfyuxqd9fTcc89VOn3u3LnnMouISNAzxmB+cyd07IJ9Phu7bWOgI/ldjYri448/rnT60qVLz2kYEZFQYMIjcH4/AWIT8Dw9BZufF+hIflXtYfslS5YAcOLECd/338nPz6dp06b+SyYiEsRM4yY4d03C88g4PDMfwvnzo5iGdfPGR9UWxSeffAJAeXm57/vvNG/enNGjR/svmYhIkDMJrXB+Px7Pfz2AZ/Z/4dwxAeOc1oAXIaHaonjggQcAWLBgAYMH14+j+yIip8N06ooZNAz70mzsWwsx19a935U1qr5OnTqxd+/eCtP27t3L+vXr/RJKRCSUmAHXYXpdjn39RezGNYGOc87VqChmz55No0aNKkxr2LAhs2fP9ksoEZFQYozB3DwKWrXFM+sxbGH+j78ohNSoKA4ePEh0dHSFadHR0RQXF/sjk4hIyDENGuDccS94TuB55q/Y8vJARzpnalQUCQkJbNxY8VzhTZs2ER8f75dQIiKhyCS0wrn1Lvj3Nuw/5wc6zjlTo1GtBg0axPTp07niiitISEhg//79fPjhh4waNcrf+UREQoq5pA/mJ1dj312EvaAL5uJLAh3prNVoiyI9PZ2JEydy9OhR1qxZw9GjR7nvvvtIT0/3dz4RkZBjbhwO57XDM+e/6sQNj2o8Tm5ycrKGGhcRqQET2QDnt+PwPDQWz7z/xrlzIsaYQMc6YzUqiuPHj/PKK6+wbNkyDh06xLx58/j888/Jy8vj6quvPqsACxYsYNWqVRhjaN68OaNGjcLlcmGtZc6cOaxdu5YGDRowatQokpKSzmpZIiK1xZzXDvPLW7EvzcJ+8i6m/9n9rgykGu16mjdvHrt27WLMmDG+VmzTpg3vvffeWQe47rrrmD59Oo8++ig9evTglVdeAWDt2rXs27ePGTNmMHLkSGbNmnXWyxIRqU3mimuhU1fswjnYgtC9f0+NimLFihWMGTOG1NRUX1G4XC7cbvdZB4iK+n5slGPHjvnmv2rVKvr3748xhtTUVEpLSykqKjrr5YmI1BbjODi3jgEDnrkzsB5PoCOdkRrtegoPD8fzgzdYUlJyzgYFfPHFF1m6dClRUVG+YUPcbjexsbG+58TExOB2u0+5ngNg8eLFLF68GICpU6dWeN3J76Gy6cFMmWuHMvtfqOWFc5g5NpYjw/9ASfYUGq/5lKirbzj7eVbBX+u5RkXRq1cvZs6cyW233QZAUVERc+fOJSMjo0YLycrKqvTivMGDB5Oens6QIUMYMmQIixYt4p133uHGG2+s8RsAyMzMJDMz0/dzQUHBKc+JjY2tdHowU+baocz+F2p54dxmtl17QaeuHJqXTWnShRiXf0rzbDK3atWqysdqVBRDhw7l73//O3fffTdlZWWMGTOGAQMGMGjQoBoFmDRpUo2e169fP6ZMmcKNN96Iy+Wq8IYLCwtxuVw1mo+ISDAxxuDcMhrP5Dvx/O//4Iy+L6TOgqrxrqfbbruN2267zbfL6Vy9yby8PFq2bAnAypUrfa2WlpbGO++8Q58+fdi+fTtRUVGV7nYSEQkFJi4Rc91N2FfmwLrl0L1XoCPVWJVFkZ+f7xuiY//+ikfrjxw58v0MwsNp0aIFYWFhZxTghRdeIC8vD2MMsbGxjBw5EoDu3buzZs0axowZQ2RkpK4CF5GQZwb8DPvZEjwLnsXp1BXTsNGPvygIVFkU48aN4/nnnwdgzJgx1c4kMjKSm2666YyuqRg3blyl040xjBgx4rTnJyISrEx4OM7Nd+CZNgH75kuYX90W6Eg1UmVRfFcSAC+99FKVM7DW8vXXX/Pggw+e9cV3IiJ1nUm+EJMxALv4dWy/qzAJVR9EDhandc8+t9tNbm5uhesnjDG0a9fOt8tIRESqZ274DURE4HkpNC4krtHB7IKCAmbMmMG2bdto0qQJhw8fJjU1lbvuuou4uDgALr30Ur8GFRGpK0zzaMy1g7GvzMFuWou5qHugI1WrRlsU2dnZJCUlMXfuXGbNmsXcuXNJSkoiOzvb3/lEROokc8W1EJeI5+XnsJ4TgY5TrRoVxY4dO7j55ptp2LAh4L0N6s0338yOHTv8Gk5EpK4yERE4N/wG9uzELvsg0HGqVaOiSElJITc3t8K0L7/8ktTUVL+EEhGpFy7pAx1Ssa+/iC07Fug0VaryGMXJZzolJCQwZcoUevToQUxMDIWFhaxdu5a+ffvWSkgRkbrIGINzw2/wPDYR+9HbmKuuD3SkSlVZFIWFhRV+/u5gdUlJCREREfTs2ZOysjL/phMRqeNMxy5wYTfs2y9j+1+FaRj14y+qZVUWha6EFhGpHc71t+B55G7skv/DDKzZGHq1qca3Qs3Ly2PZsmW43W5cLhd9+vTxjdEkIiJnznRIgYvTsO/9E3vFNUG3VVGjg9mrVq1iwoQJ7NmzhyZNmrB3714mTJjAqlWr/J1PRKRecH42BEoPYT98K9BRTlGjLYoXX3yRe+65h86dO/umbdq0ieeee460tDS/hRMRqS9MhxS4qDv2/dewA36GiWwQ6Eg+NdqicLvddOrUqcK0jh07nnLAW0REzpzzH4Pg0MGgu66iRkXRvn173njjjQrT3nzzTdq3b++PTCIi9VPqRZB0Afa9RUF1tXaNdj2NGDGCadOm8fbbb/uuo4iMjGT8+PH+ziciUm8YY3B++gs8T0/13tyoR81uN+1vNSqK8847jyeeeILt27f7znpKTk4mPLzGJ02JiEhNdLsUYhPwvP8aYaFUFABhYWF07NjRn1lEROo944R574T30izsV9sx7VMCHen07kchIiL+ZzIGQIOGQXOqrIpCRCTImKjGmF6XYVd+gj1cEug4KgoRkWBkLr8GjpdhcwJ/qqyKQkQkCJnz2sH5HbGfvI+1NqBZVBQiIkHK9L0S9u2G3C8CmkNFISISpEx6P2jQCLtscUBzqChERIKUadAQk5aBXb0Meyxwd8BTUYiIBDHT+wo4egS79rOAZVBRiIgEs5SLwBWHXbE0YBFUFCIiQcw4Dia9L2xeiy09FJAMKgoRkSBn0vvDiRPY1TkBWb6KQkQk2LVNgviWKgoREamcMQbTIwO2rseWHq715asoRERCgOnR27v76fMVtb5sFYWISCholwwtXNj1KgoREamEcRzMxWmwaS22/HitLltFISISIkyXdDh6BLZvrtXlBk1RvPHGG9x4442UlHjHXrfW8txzz3HXXXcxbtw4duzYEeCEIiIB1qkrhIdjN6yq1cUGRVEUFBSwfv16YmNjfdPWrl3Lvn37mDFjBiNHjmTWrFkBTCgiEnimQUNIvhC7eV2tLjcoimLevHncdNNNGGN801atWkX//v0xxpCamkppaSlFRUUBTCkiEnjmou6wZye22F1rywyvtSVVYeXKlbhcLtq3b19hutvtrrCFERMTg9vtJjo6+pR5LF68mMWLvcPwTp06tcLrvhMeHl7p9GCmzLVDmf0v1PJC8GY+nnEZ7n/Mo8nuHTRKTq3wmL8y10pRZGVlUVxcfMr0wYMHs2jRIiZOnHhW88/MzCQzM9P3c0FBwSnPiY2NrXR6MFPm2qHM/hdqeSF4M9sm0RDVhENr/kVp57QKj51N5latWlX5WK0UxaRJkyqd/vXXX5Ofn88999wDQGFhIePHj2fKlCm4XK4Kb7iwsBCXy1UbcUVEgpZxHEi9CLt1Q60tM6C7ntq2bVvhIPXo0aOZMmUKzZo1Iy0tjXfeeYc+ffqwfft2oqKiKt3tJCJS35jUzth1y7FFhZjoGL8vL+DHKKrSvXt31qxZw5gxY4iMjGTUqFGBjiQiEhRM8oVYgC+/gLS+fl9eUBVFdna273tjDCNGjAhgGhGRINWmA0RGYr/cgqmFogiK02NFRKTmTHg4tE/BfrmlVpanohARCUGmfSrs2lEr4z6pKEREQlH7FCgvhz07/b4oFYWISAgy7ZMBsP/e7vdlqShEREJRbAJENYbd//b7olQUIiIhyBgDrdtjd3/l92WpKEREQpRpkwS7v8J6PH5djopCRCRUndcOjh0F9wG/LkZFISISokzLNt5v8nb5dTkqChGRUPVtUdi9X/t1MSoKEZEQZRo3gWYtYN8evy5HRSEiEsriW2Hz9/p1ESoKEZEQZuJbwv48vy5DRSEiEsriW8JBN/bYMb8tQkUhIhLKYuK9/7rz/bYIFYWISAgz3xVFoYpCREQqExMHgC1QUYiISGWau8A4cNDtt0WoKEREQpgJC/NeS1FU6LdlqChEREJdCxe2WEUhIiJVadYCSor9NnsVhYhIiDNNm8OhEr/NX0UhIhLqmjaHwwex1vpl9ioKEZFQ17QZlJdjj37jl9mrKEREQl1UEwBs6WG/zF5FISIS4sy3ReE5fMgv81dRiIiEuqjGANhSFYWIiFSmYSMA7NEjfpm9ikJEJNRFNgRUFCIiUpWG3xbFERWFiIhUJiISAFvmn5sXqShEREJdeAQA9niZX2avohARCXUR3qKg/LhfZq+iEBEJdX7eogj3y1xPw8KFC/nggw9o1qwZAEOGDKFHjx4ALFq0iCVLluA4DsOGDaNbt24BTCoiEpyMMWAMePwz1lPAiwLgmmuu4brrrqswbffu3eTk5PD4449TVFREVlYWTz75JI6jjSARkVMYA54Tfpl10P7WXblyJRkZGURERBAfH09iYiK5ubmBjiUiEpwcx2+jxwbFFsW7777L0qVLSUpK4je/+Q1NmjTB7XaTkpLie47L5cLtrvyesIsXL2bx4sUATJ06ldjY2FOeEx4eXun0YKbMtUOZ/S/U8kLoZT6YcQUNOqTQ1A+Za6UosrKyKC4uPmX64MGDueqqq/jVr34FwEsvvcTzzz/PqFGjTmv+mZmZZGZm+n4uKCg45TmxsbGVTg9mylw7lNn/Qi0vhGDmW+6k+VlkbtWqVZWP1UpRTJo0qUbPGzBgANOmTQO8WxCFhd/fA9btduNyufyST0REqhbwYxRFRUW+71esWEGbNm0ASEtLIycnh+PHj5Ofn09eXh7JycmBiikiUm8F/BjF3//+d7766iuMMcTFxTFy5EgA2rRpQ+/evRk7diyO4zB8+HCd8SQiEgABL4q77rqrysduuOEGbrjhhlpMIyIiP6Q/0UVEpFoqChERqZaKQkREqqWiEBGRahnrr2u+RUSkTqg3WxQTJkwIdITTpsy1Q5n9L9TygjKfrN4UhYiInBkVhYiIVKveFMXJgwaGCmWuHcrsf6GWF5T5ZDqYLSIi1ao3WxQiInJmVBQiIlKtgA8K6A+fffYZL7/8Mnv27OGRRx7h/PPPr/B4QUEBf/zjHxk0aJDvXt3r1q1jzpw5eDweBgwYwPXXXx8UmdevX88LL7xAeXk54eHh3HLLLXTu3BmAHTt2kJ2dTVlZGd27d2fYsGHem6wHMC/AokWLWLJkCY7jMGzYMLp16wYEfh2f7KuvvuJvf/sbZWVlhIWFMWLECJKTk7HWMmfOHNauXUuDBg0YNWoUSUlJAcv5Q2+//TbvvvsujuPQo0cPbr75ZqDqdR4s3njjDebPn8+sWbNo1qxZUK/n+fPns3r1asLDw0lISGDUqFE0btwYCN717PfPlq2Ddu3aZffs2WMfeOABm5ube8rj06dPt4899ph97bXXrLXWnjhxwt55551237599vjx43bcuHF2165dQZF5x44dtrCw0Fpr7c6dO+3IkSN9j02YMMFu3brVejwe+/DDD9s1a9YEPO+uXbvsuHHjbFlZmd2/f7+988477YkTJ4JiHZ8sKyvLt75Wr15tH3jgAd/3Dz/8sPV4PHbr1q323nvvDVjGH9qwYYN98MEHbVlZmbXW2uLiYmtt1es8WBw4cMA+9NBD9o477rAHDx601gb3el63bp0tLy+31lo7f/58O3/+fGtt8K7n2vhs1cldT61bt67ytn4rVqwgPj6e1q1b+6bl5uaSmJhIQkIC4eHhZGRksHLlytqKC1SduUOHDr47+7Vp04aysjKOHz9OUVERR44cITU1FWMM/fv3r9XMVeVduXIlGRkZREREEB8fT2JiIrm5uUGxjk9mjOHIkSMAfPPNN0RHRwOwatUq+vfvjzGG1NRUSktLK9xcK5Dee+89fv7znxMREQFA8+bNgarXebCYN28eN910U4Wt3WBez127diUsLAyA1NRU3G43ELzruTY+W3WyKKpy9OhRXnvtNQYNGlRhutvtJiYmxvdzTEyM73+OYLJ8+XKSkpKIiIgI2sw/zOVyuXC73UGX99Zbb2X+/PnccccdzJ8/n6FDhwLe/LEn3Zw+0DlPlpeXx5YtW/jzn//MAw884PslVdU6DwYrV67E5XLRvn37CtODeT2fbMmSJb7dS8G6nmvjsxWyxyiysrIoLi4+ZfrgwYNJT0+v9DULFy7kmmuuoWHDhn5OV7kzyfydXbt28cILL3Dffff5Kd2pziZvMKgu/4YNG7j11lvp1asXOTk5/M///E+N7+3uT9Vl9ng8HD58mIcffpgvv/ySJ554gpkzZ9Z+yB+oLvOiRYuYOHFi7Yf6ETX5f/vVV18lLCyMfv361XK64BOyRXEmH+rc3FyWL1/OCy+8QGlpKcYYIiMjSUpKorCw0Pe8wsJC3+6ec+lMfxEVFhYyffp0Ro8eTWJiIuD9a8bfmc8k7w9zud1uX67aWMcnqy7/zJkzGTZsGAC9e/fmmWeeAbz5CwoKajXnyarL/N5779GzZ0+MMSQnJ+M4DocOHap2ndeGqjJ//fXX5Ofnc8899wDedTl+/HimTJkS1OsZ4KOPPmL16tXcf//9vl1mgV7PVamN3wX1atfTgw8+SHZ2NtnZ2QwcOJBf/OIXXH311Zx//vnk5eWRn59PeXk5OTk5pKWlBTouAKWlpUydOpWhQ4fSsWNH3/To6GgaNWrEtm3bsNaydOnSoMiclpZGTk4Ox48fJz8/n7y8PJKTk4NuHbtcLjZv3gzAxo0bfQWclpbG0qVLsdaybds2oqKifMcvAi09PZ1NmzYBsHfvXsrLy2natGmV6zzQ2rZty6xZs3yfuZiYGKZNm0aLFi2Cej2vW7eO1157jfHjx9OgQQPf9GBdz7Xx2aqTV2avWLGC5557jpKSEho3bkz79u1P2WWzcOFCGjZs6Ds9ds2aNcybNw+Px8Pll19e6/fqrirzP/7xD/75z3/6fpEBTJw4kebNm/Pll1/y1FNPUVZWRrdu3bj99ttr7fTY6tbxq6++yocffojjONx22210794dCPw6PtmWLVt8pxNGREQwYsQIkpKSsNYye/ZsPv/8cyIjIxk1atQpp1cHSnl5OU899RQ7d+485VTpqtZ5MBk9ejRTpkzxnR4brOv5rrvuory8nCZNmgCQkpLCyJEjgeBdz/7+bNXJohARkXOnXu16EhGR06eiEBGRaqkoRESkWioKERGplopCRESqpaIQAbKzs1mwYEGVj99yyy3s37//nC/XX/MVOZdUFCI1MH/+fBISEs5qHpMnT+aDDz445/P9ofLych577DFGjx7NjTfe6LtIT+RMqShE6qCOHTty11130aJFi0BHkTogZMd6Ejldu3fvZtasWXz11Ve4XC6GDh1aYaiDkpISsrKy2L59Ox06dODOO+8kLi4OgBtvvJEZM2aQmJjI8ePHefHFF/nss88oLy8nPT2d2267jcjISMA7YurChQvJz8+nWbNmDB8+nC+++IIvvviC7du3M3fuXC677DKGDx/um++hQ4f461//yjPPPIPjeP9+W7FiBQsXLmT69Ol4PB5ef/11PvjgA0pLS+ncuTMjR470XT18svDwcK655hoA37xEzob+L5J6oby8nGnTptGlSxdmzZrF7bffzowZM9i7d6/vOZ9++im//OUvmT17Nu3bt2fGjBmVzuuFF14gLy+PRx99lBkzZuB2u3nllVcA78CTM2fO5JZbbmHOnDn85S9/IS4ujiFDhtCpUyduv/125s+fz/DhwyvMMyUlhYYNG7Jx48YKefr27QvAO++8w8qVK5k8eTLPPPMMTZo0YdasWed6NYlUSkUh9cL27ds5evQo119/PeHh4XTu3JkePXrw6aef+p7To0cPLrzwQiIiIhgyZAjbtm2rMMIpgLWWDz74gFtvvZUmTZrQqFEjbrjhBpYtWwZ4719w+eWX06VLFxzHweVycd5559UoY58+fXx5jhw5wtq1a+nTpw8A77//PoMHDyYmJoaIiAgGDRrE8uXLOXHixLlYPSLV0q4nqReKioqIjY2tsCsmLi6uwg1eTr75S8OGDWnSpInvdd8pKSnh2LFjTJgwwTfNWovH4wG8Qzyf6UBxffv2ZeLEifz2t79l+fLldOjQwbfr68CBA0yfPr3CoI+O43Dw4MGgGOpa6jYVhdQL0dHRFBQU4PF4fGVRUFBAy5Ytfc85eUz/o0ePcvjw4VOGvm7atCmRkZE8/vjjlf6CjomJYd++fZVm+LGRfVu3bk1cXBxr165l2bJlvt1O3833jjvuqDDUvEht0a4nqRdSUlJo0KABr7/+OuXl5WzatInVq1f7du0ArF27li1btlBeXs6CBQtITU2tsDUB3r/iBwwYwNy5czl48CDgvYHNunXrALjiiiv46KOP2LBhAx6PB7fbzZ49ewDvPa5/7JqJPn368Pbbb7N582Z69erlm37llVeyYMECDhw4AHi3bKq7L/Lx48cpKysDvMdnysrK0EDRcqY0zLjUG7t27apw1tOQIUPo2bMn4L3gLiIigv3797Nt2zaSkpIYPXo08fHxQMWznsrKynjllVfIycnx3WHuyiuvZODAgcD3Zyvl5+fTvHlzhg8fTrdu3di2bRvZ2dmUlJTQr18/br/99grzBe9WzujRo+nWrRv33nuvL7vH4+Gtt97i/fffp6ioiObNm9O7d2/fvb5/aPTo0b5S+c7MmTN970fkdKgoRH6Ex+Nh8ODBPPXUU6dsYYjUB9r1JPIjdu3aRUREhC5ek3pLB7NFqvGvf/2LZ599lptuuonwcH1cpH7SricREamWdj2JiEi1VBQiIlItFYWIiFRLRSEiItVSUYiISLX+H9ALlLo5TVbHAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(points[:,0], points[:,1])\n",
"plt.xlabel('objective 1')\n",
"plt.ylabel('objective 2')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Building the models"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"PyePAL comes with helper functions to build the Gradient Boosted Decision Tree models."
]
},
{
"cell_type": "code",
"execution_count": 86,
"metadata": {},
"outputs": [],
"source": [
"objective_0_models = build_gbdt_tuple(n_estimators=50, num_leaves=10)\n",
"objective_1_models = build_gbdt_tuple(n_estimators=50, num_leaves=10)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initializing PyePAL"
]
},
{
"cell_type": "code",
"execution_count": 92,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
" /Users/kevinmaikjablonka/Dropbox (LSMO)/Documents/open_source/PythonPAL/pyepal/pal/validate_inputs.py:117: UserWarning:Only one epsilon value provided,\n",
"will automatically expand to use the same value in every dimension\n",
" /Users/kevinmaikjablonka/Dropbox (LSMO)/Documents/open_source/PythonPAL/pyepal/pal/validate_inputs.py:145: UserWarning:No goals provided, will assume that every dimension should be maximized\n"
]
}
],
"source": [
"palinstance = PALGBDT(x, [objective_0_models, objective_1_models], 2, coef_var_threshold=10, beta_scale=1/50)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Before we can handover the work to a loop we need to initialize the `palinstance` with some measurement. Often, a diverse set is the best choice and PyePAL provides an utilities to calculate this set (`get_kmeans_samples`, `get_maxmin_samples`). Here, we will use a greedy sampling of the farthest points in design space, initialized with the mean and using the Euclidean distance as distance metric (these are the defaults)."
]
},
{
"cell_type": "code",
"execution_count": 93,
"metadata": {},
"outputs": [],
"source": [
"from pyepal import get_kmeans_samples"
]
},
{
"cell_type": "code",
"execution_count": 94,
"metadata": {},
"outputs": [],
"source": [
"indices = get_kmeans_samples(x, 5)"
]
},
{
"cell_type": "code",
"execution_count": 95,
"metadata": {},
"outputs": [],
"source": [
"palinstance.update_train_set(indices, points[indices])\n",
"palinstance.cross_validation_points = 0 # for performance reasons, we won't perform cross validation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Now we can explore the space"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To have some more custom control we will write our own loop, but we could also just use `exhaust_loop`."
]
},
{
"cell_type": "code",
"execution_count": 96,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"pyepal at iteration 2. 0 Pareto optimal points, 0 discarded points, 100 unclassified points.\n",
"pyepal at iteration 3. 0 Pareto optimal points, 0 discarded points, 100 unclassified points.\n",
"pyepal at iteration 4. 1 Pareto optimal points, 0 discarded points, 99 unclassified points.\n",
"pyepal at iteration 5. 2 Pareto optimal points, 0 discarded points, 98 unclassified points.\n",
"pyepal at iteration 6. 2 Pareto optimal points, 0 discarded points, 98 unclassified points.\n",
"pyepal at iteration 7. 3 Pareto optimal points, 0 discarded points, 97 unclassified points.\n",
"pyepal at iteration 8. 3 Pareto optimal points, 0 discarded points, 97 unclassified points.\n",
"pyepal at iteration 9. 3 Pareto optimal points, 0 discarded points, 97 unclassified points.\n",
"pyepal at iteration 10. 4 Pareto optimal points, 0 discarded points, 96 unclassified points.\n",
"pyepal at iteration 11. 4 Pareto optimal points, 0 discarded points, 96 unclassified points.\n",
"pyepal at iteration 12. 4 Pareto optimal points, 0 discarded points, 96 unclassified points.\n",
"pyepal at iteration 13. 4 Pareto optimal points, 0 discarded points, 96 unclassified points.\n",
"pyepal at iteration 14. 4 Pareto optimal points, 0 discarded points, 96 unclassified points.\n",
"pyepal at iteration 15. 4 Pareto optimal points, 0 discarded points, 96 unclassified points.\n",
"pyepal at iteration 16. 4 Pareto optimal points, 0 discarded points, 96 unclassified points.\n",
"pyepal at iteration 17. 4 Pareto optimal points, 0 discarded points, 96 unclassified points.\n",
"pyepal at iteration 18. 5 Pareto optimal points, 0 discarded points, 95 unclassified points.\n",
"pyepal at iteration 19. 6 Pareto optimal points, 0 discarded points, 94 unclassified points.\n",
"pyepal at iteration 20. 7 Pareto optimal points, 0 discarded points, 93 unclassified points.\n",
"pyepal at iteration 21. 8 Pareto optimal points, 0 discarded points, 92 unclassified points.\n",
"pyepal at iteration 22. 9 Pareto optimal points, 0 discarded points, 91 unclassified points.\n",
"pyepal at iteration 23. 10 Pareto optimal points, 0 discarded points, 90 unclassified points.\n",
"pyepal at iteration 24. 11 Pareto optimal points, 0 discarded points, 89 unclassified points.\n",
"pyepal at iteration 25. 12 Pareto optimal points, 0 discarded points, 88 unclassified points.\n",
"pyepal at iteration 26. 13 Pareto optimal points, 0 discarded points, 87 unclassified points.\n",
"pyepal at iteration 27. 14 Pareto optimal points, 0 discarded points, 86 unclassified points.\n",
"pyepal at iteration 28. 15 Pareto optimal points, 0 discarded points, 85 unclassified points.\n",
"pyepal at iteration 29. 16 Pareto optimal points, 0 discarded points, 84 unclassified points.\n",
"pyepal at iteration 30. 17 Pareto optimal points, 0 discarded points, 83 unclassified points.\n",
"pyepal at iteration 31. 18 Pareto optimal points, 0 discarded points, 82 unclassified points.\n",
"pyepal at iteration 32. 19 Pareto optimal points, 0 discarded points, 81 unclassified points.\n",
"pyepal at iteration 33. 20 Pareto optimal points, 0 discarded points, 80 unclassified points.\n",
"pyepal at iteration 34. 21 Pareto optimal points, 0 discarded points, 79 unclassified points.\n",
"pyepal at iteration 35. 22 Pareto optimal points, 0 discarded points, 78 unclassified points.\n",
"pyepal at iteration 36. 23 Pareto optimal points, 0 discarded points, 77 unclassified points.\n",
"pyepal at iteration 37. 24 Pareto optimal points, 0 discarded points, 76 unclassified points.\n",
"pyepal at iteration 38. 25 Pareto optimal points, 0 discarded points, 75 unclassified points.\n",
"pyepal at iteration 39. 42 Pareto optimal points, 0 discarded points, 58 unclassified points.\n",
"pyepal at iteration 40. 43 Pareto optimal points, 0 discarded points, 57 unclassified points.\n",
"pyepal at iteration 41. 44 Pareto optimal points, 0 discarded points, 56 unclassified points.\n",
"pyepal at iteration 42. 45 Pareto optimal points, 0 discarded points, 55 unclassified points.\n",
"pyepal at iteration 43. 46 Pareto optimal points, 0 discarded points, 54 unclassified points.\n",
"pyepal at iteration 44. 47 Pareto optimal points, 0 discarded points, 53 unclassified points.\n",
"pyepal at iteration 45. 48 Pareto optimal points, 0 discarded points, 52 unclassified points.\n",
"pyepal at iteration 46. 49 Pareto optimal points, 0 discarded points, 51 unclassified points.\n",
"pyepal at iteration 47. 50 Pareto optimal points, 0 discarded points, 50 unclassified points.\n",
"pyepal at iteration 48. 51 Pareto optimal points, 0 discarded points, 49 unclassified points.\n",
"pyepal at iteration 49. 52 Pareto optimal points, 0 discarded points, 48 unclassified points.\n",
"pyepal at iteration 50. 53 Pareto optimal points, 0 discarded points, 47 unclassified points.\n",
"pyepal at iteration 51. 54 Pareto optimal points, 0 discarded points, 46 unclassified points.\n",
"Done. No unclassified point left\n",
"pyepal at iteration 51. 100 Pareto optimal points, 0 discarded points, 0 unclassified points.\n"
]
}
],
"source": [
"while sum(palinstance.unclassified) > 0: \n",
" new_index = palinstance.run_one_step()\n",
" print(palinstance) # the string representation of the object will give basic information about the state\n",
" # if there is nothing to sample left, run_one_step() will return None\n",
" if new_index is not None: \n",
" palinstance.update_train_set(new_index, points[new_index])\n",
" else: \n",
" break"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.8"
}
},
"nbformat": 4,
"nbformat_minor": 4
}