src/go/collectors/go.d.plugin/modules/k8s_kubelet/integrations/kubelet.md
<!--startmeta
custom_edit_url: "https://github.com/netdata/netdata/edit/master/src/go/collectors/go.d.plugin/modules/k8s_kubelet/README.md"
meta_yaml: "https://github.com/netdata/netdata/edit/master/src/go/collectors/go.d.plugin/modules/k8s_kubelet/metadata.yaml"
sidebar_label: "Kubelet"
learn_status: "Published"
learn_rel_path: "Collecting Metrics/Kubernetes"
most_popular: True
message: "DO NOT EDIT THIS FILE DIRECTLY, IT IS GENERATED BY THE COLLECTOR'S metadata.yaml FILE"
endmeta-->
# Kubelet
<img src="https://netdata.cloud/img/kubernetes.svg" width="150"/>
Plugin: go.d.plugin
Module: k8s_kubelet
<img src="https://img.shields.io/badge/maintained%20by-Netdata-%2300ab44" />
## Overview
This collector monitors Kubelet instances.
This collector is supported on all platforms.
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
### Default Behavior
#### Auto-Detection
This integration doesn't support auto-detection.
#### Limits
The default configuration for this integration does not impose any limits on data collection.
#### Performance Impact
The default configuration for this integration is not expected to impose a significant performance impact on the system.
## Metrics
Metrics grouped by *scope*.
The scope defines the instance that the metric belongs to. An instance is uniquely identified by a set of labels.
### Per Kubelet instance
These metrics refer to the entire monitored application.
This scope has no labels.
Metrics:
| Metric | Dimensions | Unit |
|:------|:----------|:----|
| k8s_kubelet.apiserver_audit_requests_rejected | rejected | requests/s |
| k8s_kubelet.apiserver_storage_data_key_generation_failures | failures | events/s |
| k8s_kubelet.apiserver_storage_data_key_generation_latencies | 5_µs, 10_µs, 20_µs, 40_µs, 80_µs, 160_µs, 320_µs, 640_µs, 1280_µs, 2560_µs, 5120_µs, 10240_µs, 20480_µs, 40960_µs, +Inf | observes/s |
| k8s_kubelet.apiserver_storage_data_key_generation_latencies_percent | 5_µs, 10_µs, 20_µs, 40_µs, 80_µs, 160_µs, 320_µs, 640_µs, 1280_µs, 2560_µs, 5120_µs, 10240_µs, 20480_µs, 40960_µs, +Inf | percentage |
| k8s_kubelet.apiserver_storage_envelope_transformation_cache_misses | cache misses | events/s |
| k8s_kubelet.kubelet_containers_running | total | running_containers |
| k8s_kubelet.kubelet_pods_running | total | running_pods |
| k8s_kubelet.kubelet_pods_log_filesystem_used_bytes | a dimension per namespace and pod | B |
| k8s_kubelet.kubelet_runtime_operations | a dimension per operation type | operations/s |
| k8s_kubelet.kubelet_runtime_operations_errors | a dimension per operation type | errors/s |
| k8s_kubelet.kubelet_docker_operations | a dimension per operation type | operations/s |
| k8s_kubelet.kubelet_docker_operations_errors | a dimension per operation type | errors/s |
| k8s_kubelet.kubelet_node_config_error | experiencing_error | bool |
| k8s_kubelet.kubelet_pleg_relist_interval_microseconds | 0.5, 0.9, 0.99 | microseconds |
| k8s_kubelet.kubelet_pleg_relist_latency_microseconds | 0.5, 0.9, 0.99 | microseconds |
| k8s_kubelet.kubelet_token_requests | total, failed | token_requests/s |
| k8s_kubelet.rest_client_requests_by_code | a dimension per HTTP status code | requests/s |
| k8s_kubelet.rest_client_requests_by_method | a dimension per HTTP method | requests/s |
### Per volume manager
These metrics refer to the Volume Manager.
This scope has no labels.
Metrics:
| Metric | Dimensions | Unit |
|:------|:----------|:----|
| k8s_kubelet.volume_manager_total_volumes | actual, desired | state |
## Alerts
The following alerts are available:
| Alert name | On metric | Description |
|:------------|:----------|:------------|
| [ kubelet_node_config_error ](https://github.com/netdata/netdata/blob/master/src/health/health.d/kubelet.conf) | k8s_kubelet.kubelet_node_config_error | the node is experiencing a configuration-related error (0: false, 1: true) |
| [ kubelet_token_requests ](https://github.com/netdata/netdata/blob/master/src/health/health.d/kubelet.conf) | k8s_kubelet.kubelet_token_requests | number of failed Token() requests to the alternate token source |
| [ kubelet_token_requests ](https://github.com/netdata/netdata/blob/master/src/health/health.d/kubelet.conf) | k8s_kubelet.kubelet_token_requests | number of failed Token() requests to the alternate token source |
| [ kubelet_operations_error ](https://github.com/netdata/netdata/blob/master/src/health/health.d/kubelet.conf) | k8s_kubelet.kubelet_operations_errors | number of Docker or runtime operation errors |
| [ kubelet_operations_error ](https://github.com/netdata/netdata/blob/master/src/health/health.d/kubelet.conf) | k8s_kubelet.kubelet_operations_errors | number of Docker or runtime operation errors |
## Setup
### Prerequisites
No action required.
### Configuration
#### File
The configuration file name for this integration is `go.d/k8s_kubelet.conf`.
You can edit the configuration file using the `edit-config` script from the
Netdata [config directory](/docs/netdata-agent/configuration/README.md#the-netdata-config-directory).
```bash
cd /etc/netdata 2>/dev/null || cd /opt/netdata/etc/netdata
sudo ./edit-config go.d/k8s_kubelet.conf
```
#### Options
The following options can be defined globally: update_every, autodetection_retry.
<details><summary>Config options</summary>
| Name | Description | Default | Required |
|:----|:-----------|:-------|:--------:|
| update_every | Data collection frequency. | 1 | no |
| autodetection_retry | Recheck interval in seconds. Zero means no recheck will be scheduled. | 0 | no |
| url | Server URL. | http://127.0.0.1:10255/metrics | yes |
| timeout | HTTP request timeout. | 1 | no |
| username | Username for basic HTTP authentication. | | no |
| password | Password for basic HTTP authentication. | | no |
| proxy_url | Proxy URL. | | no |
| proxy_username | Username for proxy basic HTTP authentication. | | no |
| proxy_password | Password for proxy basic HTTP authentication. | | no |
| method | HTTP request method. | GET | no |
| body | HTTP request body. | | no |
| headers | HTTP request headers. | | no |
| not_follow_redirects | Redirect handling policy. Controls whether the client follows redirects. | no | no |
| tls_skip_verify | Server certificate chain and hostname validation policy. Controls whether the client performs this check. | no | no |
| tls_ca | Certification authority that the client uses when verifying the server's certificates. | | no |
| tls_cert | Client TLS certificate. | | no |
| tls_key | Client TLS key. | | no |
</details>
#### Examples
##### Basic
A basic example configuration.
```yaml
jobs:
- name: local
url: http://127.0.0.1:10255/metrics
```
##### HTTPS with self-signed certificate
Do not validate server certificate chain and hostname.
<details><summary>Config</summary>
```yaml
jobs:
- name: local
url: http://127.0.0.1:10250/metrics
tls_skip_verify: yes
```
</details>
## Troubleshooting
### Debug Mode
To troubleshoot issues with the `k8s_kubelet` collector, run the `go.d.plugin` with the debug option enabled. The output
should give you clues as to why the collector isn't working.
- Navigate to the `plugins.d` directory, usually at `/usr/libexec/netdata/plugins.d/`. If that's not the case on
your system, open `netdata.conf` and look for the `plugins` setting under `[directories]`.
```bash
cd /usr/libexec/netdata/plugins.d/
```
- Switch to the `netdata` user.
```bash
sudo -u netdata -s
```
- Run the `go.d.plugin` to debug the collector:
```bash
./go.d.plugin -d -m k8s_kubelet
```