docs/resources/kubelet_pool.md
# Kubelet Pool Resource
This resource allows to create one or more kubelet containers on remote hosts over SSH using Docker container runtime.
## Example Usage
```hcl
resource "flexkube_pki" "pki" {
kubernetes {}
}
resource "flexkube_kubelet_pool" "workers" {
bootstrap_config {
server = "api.example.com"
token = "bootstrap.token"
}
pki_yaml = flexkube_pki.pki.state_yaml
cgroup_driver = "cgroupfs"
hairpin_mode = "hairpin-veth"
volume_plugin_dir = "/var/lib/kubelet/volumeplugins"
cluster_dns_ips = [
"11.0.0.10"
]
system_reserved = {
"cpu" = "100m"
"memory" = "500Mi"
}
kube_reserved = {
"memory" = "100Mi"
"cpu" = "100m"
}
ssh {
user = "core"
port = 22
}
kubelet {
name = "foo"
address = "10.0.0.10"
host {
ssh {
address = "10.0.0.10"
}
}
}
}
```
## Argument Reference
* `bootstrap_config` - (Required) A `client` block as defined below. This block defines parameters for kubelet bootstrapping process.
* `cluster_dns_ips` - (Required) List of cluster DNS IP addresses, which will be set in the pods.
* `cgroup_driver` - (Required) Cgroup driver to be used by kubelet. Please refer to your distribution documentation to see the right choice for you. Valid values are `cgroupfs` and `systemd`. Docker runtime on the node must have the same driver configured.
* `hairpin_mode` - (Required) Hairpin mode to be used by kubelet. Valid values are `promiscuous-bridge` and `hairpin-veth`.
* `volume_plugin_dir` - (Required) Path where kubelet should install volume plugins. For OSes like Flatcar Linux, it should be set to `/var/lib/kubelet/volumeplugins`. For other OSes it can be set to `/usr/libexec/kubernetes/kubelet-plugins/volume/exec`. This setting should be in sync with kube-controller-manager.
* `kubelet` - (Required) A `kubelet` block as defined below. This block defines single kubelet instance and can be specified multiple times.
* `system_reserved` - (Optional) Key-value map of resources to be reserved by kubelet for the OS operation. See [Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#system-reserved) documentation for more details.
* `kube_reserved` - (Optional) Key-value map of resources to be reserved by kubelet for the Kubernetes operations See [Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#kube-reserved) documentation for more details.
* `image` - (Optional) Docker image with tag to be used to run kubelet container. Defaults to `libflexkube` [default Kubernetes Image](https://github.com/flexkube/libflexkube/blob/master/pkg/defaults/defaults.go#L9).
* `ssh` - (Optional) A `ssh` block as defined below. This block defines global SSH settings shared by all instances.
* `kubernetes_ca_certificate` - (Optional) If `pki_yaml` field is not set, this field must contain PEM encoded Kubernetes X.509 CA certificate.
* `taints` - (Optional) Key-value map of taints to be applied on the node while it registers to the cluster.
* `labels` - (Optional) Key-value map of labels to be applied on the node while it registers to the cluster. Please note, that some labels cannot be applied using this field, for example labels responsible for node isolation like `node-role.kubernetes.io`. This labels must be applied via `privileged_labels` field.
* `privileged_labels` - (Optional) Key-value map of labels, which cannot be applied to the Node object by kubelet itself, like `node-role.kubernetes.io`. Those labels will be used to patch the Node object after it registers into the API server.
* `admin_config` - (Optional) A `client` block as defined below. This config will be used for applying labels from `privileged_labels` field. Please note, that Kubernetes API address defined in this block must be reachable from the machine where Terraform runs.
* `pki_yaml` - (Optional) This field can be set to `state_yaml` value of `flexkube_pki` resource to enable PKI integration for kubelet, which allows automatic use of certificates generated by PKI resource.
* `wait_for_node_ready` - (Optional) If set to `true`, resource will wait for Node object to become ready in the cluster before proceeding. Defaults to `false`.
* `extra_mount` - (Optional) A `mount` block as defined below. Can be specified zero or more times. Each block represents extra mount to be added to all kubelet containers managed by this pool.
* `extra_args` - (Optional) A list of extra arguments to pass to kubelet container.
---
A `client` block supports the following:
* `server` - (Required) Address of Kubernetes API server to connect to.
* `ca_certificate` - (Optional) PEM encoded X.509 Kubernetes CA certificate used to validate the connection to the API server.
* `client_certificate` - (Optional) PEM encoded X.509 client certificate to use for authentication to the API server.
* `client_key` - (Optional) PEM encoded private key matching certificate specified in `client_certificate` to use for authentication to the API server.
* `token` - (Optional) Bearer token to use for autenthication to the API server. Usually used by kubelet while bootstrapping.
---
A `kubelet` block supports the following:
* `name` - (Required) Name of the node.
* `bootstrap_config` - (Required) A `bootstrap_config` block as defined below. This block defines parameters for kubelet bootstrapping process.
* `cluster_dns_ips` - (Required) List of cluster DNS IP addresses, which will be set in the pods.
* `cgroup_driver` - (Required) Cgroup driver to be used by kubelet. Please refer to your distribution documentation to see the right choice for you. Valid values are `cgroupfs` and `systemd`. Docker runtime on the node must have the same driver configured.
* `hairpin_mode` - (Required) Hairpin mode to be used by kubelet. Valid values are `promiscuous-bridge` and `hairpin-veth`.
* `volume_plugin_dir` - (Required) Path where kubelet should install volume plugins. For OSes like Flatcar Linux, it should be set to `/var/lib/kubelet/volumeplugins`. For other OSes it can be set to `/usr/libexec/kubernetes/kubelet-plugins/volume/exec`. This setting should be in sync with kube-controller-manager.
* `system_reserved` - (Optional) Key-value map of resources to be reserved by kubelet for the OS operation. See [Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#system-reserved) documentation for more details.
* `kube_reserved` - (Optional) Key-value map of resources to be reserved by kubelet for the Kubernetes operations See [Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#kube-reserved) documentation for more details.
* `image` - (Optional) Docker image with tag to be used to run kubelet container. Defaults to `libflexkube` [default Kubernetes Image](https://github.com/flexkube/libflexkube/blob/master/pkg/defaults/defaults.go#L9).
* `host` - (Optional) A `host` block as defined below. This block defines where to connect for creating the container.
* `kubernetes_ca_certificate` - (Optional) If `pki_yaml` field is not set, this field must contain PEM encoded Kubernetes X.509 CA certificate.
* `taints` - (Optional) Key-value map of taints to be applied on the node while it registers to the cluster.
* `labels` - (Optional) Key-value map of labels to be applied on the node while it registers to the cluster. Please note, that some labels cannot be applied using this field, for example labels responsible for node isolation like `node-role.kubernetes.io`. This labels must be applied via `privileged_labels` field.
* `privileged_labels` - (Optional) Key-value map of labels, which cannot be applied to the Node object by kubelet itself, like `node-role.kubernetes.io`. Those labels will be used to patch the Node object after it registers into the API server.
* `admin_config` - (Optional) This kubeconfig file will be used for applying labels from `privileged_labels` field. Please note, that Kubernetes API address defined in this file must be reachable from the machine where Terraform runs.
* `wait_for_node_ready` - (Optional) If set to `true`, resource will wait for Node object to become ready in the cluster before proceeding. Defaults to `false`.
* `extra_mount` - (Optional) A `mount` block as defined below. Can be specified zero or more times. Each block represents extra mount to be added to all kubelet containers managed by this pool.
* `extra_args` - (Optional) A list of extra arguments to pass to kubelet container.
---
A `host` block supports the following:
* `direct` - (Optional) A `direct` block as defined below. Mutually exclusive with all other fields in this block. If defined, container will be created on local machine.
* `ssh` - (Optional) A `ssh` block as defined below. Mutually exclusive with all other fields in this block. If defined, container will be created on a remote machine using SSH connection.
---
A `direct` block does not support any arguments.
---
A `ssh` block supports the following:
* `address` - (Required) An address where SSH client should connect to. Can be either hostname of IP address.
* `port` - (Optional) Port where to open SSH connection. Defaults to `22`.
* `user` - (Optional) Username to use when opening SSH connection. Defaults to `root`.
* `password` - (Optional) Password to use for SSH authentication. Can be combined with `private_key` and SSH agent authentication methods.
* `connection_timeout` - (Optional) Duration for how long to wait before connection attempts times out, expressed in [Go Duration format](https://golang.org/pkg/time/#ParseDuration). Defaults to `30s`.
* `retry_timeout` - (Optional) Duration for how long to wait before giving up on connection attempts, expressed in [Go Duration format](https://golang.org/pkg/time/#ParseDuration). Defaults to `60s`.
* `retry_interval` - (Optional) Duration for how long to wait between connection attempts, expressed in [Go Duration format](https://golang.org/pkg/time/#ParseDuration). Defaults to `1s`.
* `private_key` - (Optional) PEM encoded privat key to be used for authentication. Can be combined with `password` and SSH agent authentication methods.
---
A `mount` block supports the following:
* `source` - (Required) A host directory to be mounted in the container.
* `target` - (Required) Path in the container where host directory should be mounted.
* `propagation` - (Optional) Propagation mode to be set on the mount. See [Docker](https://docs.docker.com/storage/bind-mounts/#configure-bind-propagation) documentation for allowed values.
## Attribute Reference
* `state` - A list of `host_configured_container` blocks as defined below. This attribute represents generated configuration of the managed containers. Sensitive values like configuration files content, environment variables or SSH password are replaced with SHA256 of the values. To get the actual value, use `state_sensitive` block.
* `state_sensitive` - A list of `host_configured_container` blocks as defined below. This attribute represents generated configuration of the managed containers. This attribute is marked entirely as sensitive, so it won't show up detailed in a plan. To see specific changes in generated configuration, use `state` attribute.
* `state_yaml` - State of created containers in YAML format. Can be dumped to a `state.yaml` file and used together with `flexkube kubelet-pool` command.
* `config_yaml` - Generated configuration in YAML format, which can be used by the `flexkube kubelet-pool` command.
---
A `host_configured_container` block supports the following:
* `name` - Name of the container.
* `container` - A `container` block as defined below. Contains container configuration parameters.
* `config_files` - A map of configuration files which are created for the container on the host. The key is the path on the host to the configuration file and the value is either a configuration file content or it's SHA256, depending if read from `state` or from `state_sensitive` attribute.
* `host` - A `host` block as defined above. Describes on which host the container is created.
---
A `container` block supports the following:
* `config` - A `config` block as defined below. Contains container configuration attributes.
* `runtime` - A `runtime` block as defined below. Includes container runtime configuration.
* `status` - A `status` block as defined below. Contains container status information.
---
A `config` block supports the following:
* `name` - Name of the managed container.
* `image` - Container image used.
* `privileged` - If `true`, the container runs as a privileged process on the host.
* `args` - Arguments used for a container.
* `entrypoint` - Binary name which runs in the container.
* `port` - A `port` block as defined below. Contains ports, which are exposed by the container.
* `mount` - A list of `mount` blocks as defined above. Contains information which host paths are mounted into the container.
* `network_mode` - Defines what network mode container use. Actual value may depend on used container runtime.
* `pid_mode` - Defines in which PID mode container runs. Actual value may depend on used container runtime.
* `ipc_mode` - Defines in which IPC mode container runs. Actual value may depend on used container runtime.
* `user` - Name of the user or UID used by the container.
* `group` - Name of the group or GID used by the container.
---
A `port` block supports the following:
* `ip` - IP on which the port is exposed.
* `port` - Exposed port number.
* `protocol` - Exposed protocol.
---
A `runtime` block supports the following:
* `docker` - A `docker` block as defined below. Container Docker runtime configuration attributes.
---
A `docker` block supports the following:
* `host` - URL used to talk to Docker runtime. Defaults to `unix:///var/run/docker.sock`.
---
A `status` block supports the following:
* `id` - ID of the created container given by used container runtime.
* `status` - Text status of the container. If field is empty, it means that the container does not exist.