docs/ipynb/customized.ipynb
{
"cells": [
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab_type": "code"
},
"outputs": [],
"source": [
"!pip install autokeras"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab_type": "code"
},
"outputs": [],
"source": [
"import keras\n",
"import numpy as np\n",
"import tree\n",
"from keras.datasets import mnist\n",
"\n",
"import autokeras as ak"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text"
},
"source": [
"In this tutorial, we show how to customize your search space with\n",
"[AutoModel](/auto_model/#automodel-class) and how to implement your own block\n",
"as search space. This API is mainly for advanced users who already know what\n",
"their model should look like.\n",
"\n",
"## Customized Search Space\n",
"First, let us see how we can build the following neural network using the\n",
"building blocks in AutoKeras.\n",
"\n",
"<div class=\"mermaid\">\n",
"graph LR\n",
" id1(ImageInput) --> id2(Normalization)\n",
" id2 --> id3(Image Augmentation)\n",
" id3 --> id4(Convolutional)\n",
" id3 --> id5(ResNet V2)\n",
" id4 --> id6(Merge)\n",
" id5 --> id6\n",
" id6 --> id7(Classification Head)\n",
"</div>\n",
"\n",
"We can make use of the [AutoModel](/auto_model/#automodel-class) API in\n",
"AutoKeras to implemented as follows.\n",
"The usage is the same as the [Keras functional\n",
"API](https://www.tensorflow.org/guide/keras/functional).\n",
"Since this is just a demo, we use small amount of `max_trials` and `epochs`.\n"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab_type": "code"
},
"outputs": [],
"source": [
"input_node = ak.ImageInput()\n",
"output_node = ak.Normalization()(input_node)\n",
"output_node1 = ak.ConvBlock()(output_node)\n",
"output_node2 = ak.ResNetBlock(version=\"v2\")(output_node)\n",
"output_node = ak.Merge()([output_node1, output_node2])\n",
"output_node = ak.ClassificationHead()(output_node)\n",
"\n",
"auto_model = ak.AutoModel(\n",
" inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text"
},
"source": [
"Whild building the model, the blocks used need to follow this topology:\n",
"`Preprocessor` -> `Block` -> `Head`. `Normalization` and `ImageAugmentation`\n",
"are `Preprocessor`s.\n",
"`ClassificationHead` is `Head`. The rest are `Block`s.\n",
"\n",
"In the code above, we use `ak.ResNetBlock(version='v2')` to specify the version\n",
"of ResNet to use. There are many other arguments to specify for each building\n",
"block. For most of the arguments, if not specified, they would be tuned\n",
"automatically. Please refer to the documentation links at the bottom of the\n",
"page for more details.\n",
"\n",
"Then, we prepare some data to run the model.\n"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab_type": "code"
},
"outputs": [],
"source": [
"(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
"print(x_train.shape) # (60000, 28, 28)\n",
"print(y_train.shape) # (60000,)\n",
"print(y_train[:3]) # array([7, 2, 1], dtype=uint8)\n",
"\n",
"# Feed the AutoModel with training data.\n",
"auto_model.fit(x_train[:100], y_train[:100], epochs=1)\n",
"# Predict with the best model.\n",
"predicted_y = auto_model.predict(x_test)\n",
"# Evaluate the best model with testing data.\n",
"print(auto_model.evaluate(x_test, y_test))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text"
},
"source": [
"For multiple input nodes and multiple heads search space, you can refer to\n",
"[this section](/tutorial/multi/#customized-search-space).\n",
"\n",
"## Validation Data\n",
"If you would like to provide your own validation data or change the ratio of\n",
"the validation data, please refer to the Validation Data section of the\n",
"tutorials of [Image\n",
"Classification](/tutorial/image_classification/#validation-data), [Text\n",
"Classification](/tutorial/text_classification/#validation-data),\n",
"[Multi-task and Multiple Validation](/tutorial/multi/#validation-data).\n",
"\n",
"## Data Format\n",
"You can refer to the documentation of\n",
"[ImageInput](/node/#imageinput-class),\n",
"[TextInput](/node/#textinput-class),\n",
"[RegressionHead](/block/#regressionhead-class),\n",
"[ClassificationHead](/block/#classificationhead-class),\n",
"for the format of different types of data.\n",
"You can also refer to the Data Format section of the tutorials of\n",
"[Image Classification](/tutorial/image_classification/#data-format),\n",
"[Text Classification](/tutorial/text_classification/#data-format).\n",
"\n",
"## Implement New Block\n",
"\n",
"You can extend the [Block](/base/#block-class)\n",
"class to implement your own building blocks and use it with\n",
"[AutoModel](/auto_model/#automodel-class).\n",
"\n",
"The first step is to learn how to write a build function for\n",
"[KerasTuner](https://keras-team.github.io/keras-tuner/#usage-the-basics). You\n",
"need to override the [build function](/base/#build-method) of the block. The\n",
"following example shows how to implement a single Dense layer block whose\n",
"number of neurons is tunable.\n"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab_type": "code"
},
"outputs": [],
"source": [
"class SingleDenseLayerBlock(ak.Block):\n",
" def build(self, hp, inputs=None):\n",
" # Get the input_node from inputs.\n",
" input_node = tree.flatten(inputs)[0]\n",
" layer = keras.layers.Dense(\n",
" hp.Int(\"num_units\", min_value=32, max_value=512, step=32)\n",
" )\n",
" output_node = layer(input_node)\n",
" return output_node"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text"
},
"source": [
"You can connect it with other blocks and build it into an\n",
"[AutoModel](/auto_model/#automodel-class).\n"
]
},
{
"cell_type": "code",
"execution_count": 0,
"metadata": {
"colab_type": "code"
},
"outputs": [],
"source": [
"# Build the AutoModel\n",
"input_node = ak.Input()\n",
"output_node = SingleDenseLayerBlock()(input_node)\n",
"output_node = ak.RegressionHead()(output_node)\n",
"auto_model = ak.AutoModel(input_node, output_node, overwrite=True, max_trials=1)\n",
"# Prepare Data\n",
"num_instances = 100\n",
"x_train = np.random.rand(num_instances, 20).astype(np.float32)\n",
"y_train = np.random.rand(num_instances, 1).astype(np.float32)\n",
"x_test = np.random.rand(num_instances, 20).astype(np.float32)\n",
"y_test = np.random.rand(num_instances, 1).astype(np.float32)\n",
"# Train the model\n",
"auto_model.fit(x_train, y_train, epochs=1)\n",
"print(auto_model.evaluate(x_test, y_test))"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text"
},
"source": [
"## Reference\n",
"\n",
"[AutoModel](/auto_model/#automodel-class)\n",
"\n",
"**Nodes**:\n",
"[ImageInput](/node/#imageinput-class),\n",
"[Input](/node/#input-class),\n",
"[TextInput](/node/#textinput-class).\n",
"\n",
"**Preprocessors**:\n",
"[FeatureEngineering](/block/#featureengineering-class),\n",
"[ImageAugmentation](/block/#imageaugmentation-class),\n",
"[LightGBM](/block/#lightgbm-class),\n",
"[Normalization](/block/#normalization-class),\n",
"\n",
"**Blocks**:\n",
"[ConvBlock](/block/#convblock-class),\n",
"[DenseBlock](/block/#denseblock-class),\n",
"[Merge](/block/#merge-class),\n",
"[ResNetBlock](/block/#resnetblock-class),\n",
"[RNNBlock](/block/#rnnblock-class),\n",
"[SpatialReduction](/block/#spatialreduction-class),\n",
"[TemporalReduction](/block/#temporalreduction-class),\n",
"[XceptionBlock](/block/#xceptionblock-class),\n",
"[ImageBlock](/block/#imageblock-class),\n",
"[TextBlock](/block/#textblock-class).\n"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "customized",
"private_outputs": false,
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.0"
}
},
"nbformat": 4,
"nbformat_minor": 0
}