⎈ k8s knowledge compiler

Native Histogram Support for Kubernetes Metrics [page]deterministic

reference

Kubernetes components can expose histogram metrics in [Prometheus Native Histogram](https://prometheus.io/docs/specs/native_histograms/) format, alongside the classic histogram format. Native histograms use exponential bucket boundaries instead of fixed boundaries, providing significant storage efficiency, improved query performance, and finer-grained visibility into distributions.

## Before you begin

To use native histograms, you need:

  • Kubernetes v1.36 or later with the `NativeHistograms` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) enabled.
  • Prometheus 2.40 or later to scrape and store native histograms. Prometheus 3.0+ is recommended for per-job configuration.

## What are native histograms?

Classic Prometheus histograms use fixed bucket boundaries (for example, `[0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]` seconds). Each bucket creates a separate time series (`_bucket`, `_count`, `_sum`), which can lead to:

  • High storage costs at scale, because each histogram generates many time series.
  • Accuracy issues, because data points within a wide bucket range are indistinguishable. For example, a request completing in 1µs and one completing in 4ms both fall into the same `le="0.005"` bucket.

[Native histograms](https://prometheus.io/docs/specs/native_histograms/) address these limitations by using exponential bucket boundaries that automatically adjust to the data distribution. Benefits include:

  • ~10x reduction in time series count per histogram metric, significantly reducing Prometheus storage and improving query performance.
  • Finer-grained resolution for detecting performance regressions and setting precise SLO thresholds.

## How it works

When the `NativeHistograms` feature gate is enabled, Kubernetes components expose histogram metrics in both classic and native formats simultaneously (dual exposition). The format returned depends on the `Accept` header in the HTTP request ([Prometheus content negotiation](https://prometheus.io/docs/instrumenting/exposition_formats/#content-negotiation)). Prometheus sets this header automatically based on your scrape configuration; you only need to be aware of it when querying the `/metrics` endpoint directly.

  • Text format (Accept: `text/plain`, OpenMetrics 1.0): Returns only classic histogram buckets. Backward compatible with all existing tooling. ``` # Classic histogram buckets (always present) apiserver_request_duration_seconds_bucket{le="0.005"} 1000 apiserver_request_duration_seconds_bucket{le="0.01"} 2000 ... apiserver_request_duration_seconds_bucket{le="+Inf"} 10000 apiserver_request_duration_seconds_count 10000 apiserver_request_duration_seconds_sum 450.5 ```
  • Protobuf format (Accept: `application/vnd.google.protobuf`): Contains both classic buckets and native histogram data. Prometheus automatically requests this format when `scrape_native_histograms: true` is set in the [Prometheus scrape configuration](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) for the corresponding scrape job.

This dual exposition strategy ensures:

  • Existing dashboards and alerts continue to work without changes.
  • Users can migrate queries to native histograms at their own pace.
  • Prometheus stores whichever format it is configured to ingest.

## Enabling native histograms

Enabling native histograms is a two-step process: enable the feature gate on Kubernetes components, and configure Prometheus to scrape native histograms.

### Step 1: Enable the Kubernetes feature gate

Enable the `NativeHistograms` feature gate on the Kubernetes components you want to expose native histograms from:

```bash --feature-gates=NativeHistograms=true ```

This feature gate applies to the following components: - kube-apiserver - kube-controller-manager - kube-scheduler - kubelet - kube-proxy

Each component's metrics are independent; you can enabl …(trimmed)

Sources

reference/instrumentation/native-histograms.md · docNative Histogram Support for Kubernetes Metrics

Related (10)

part_of Before you begindescribes conf=1
part_of What are native histograms?describes conf=1
part_of How it worksdescribes conf=1
part_of Enabling native histogramsdescribes conf=1
part_of Migrating dashboards and alertsdescribes conf=1
part_of Disabling native histogramsdescribes conf=1
part_of Troubleshootingdescribes conf=1
part_of Referencesdescribes conf=1
part_of Step 1: Enable the Kubernetes feature gatedescribes conf=1
part_of Step 2: Configure Prometheusdescribes conf=1

← all Docs