Pod Overhead [page]deterministic
When you run a Pod on a Node, the Pod itself takes an amount of system resources. These resources are additional to the resources needed to run the container(s) inside the Pod. In Kubernetes, _Pod Overhead_ is a way to account for the resources consumed by the Pod infrastructure on top of the container requests & limits.
In Kubernetes, the Pod's overhead is set at [admission](/docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-admission-webhooks) time according to the overhead associated with the Pod's [RuntimeClass](/docs/concepts/containers/runtime-class/).
A pod's overhead is considered in addition to the sum of container resource requests when scheduling a Pod. Similarly, the kubelet will include the Pod overhead when sizing the Pod cgroup, and when carrying out Pod eviction ranking.
## Configuring Pod overhead {#set-up}
You need to make sure a `RuntimeClass` is utilized which defines the `overhead` field.
## Usage example
To work with Pod overhead, you need a RuntimeClass that defines the `overhead` field. As an example, you could use the following RuntimeClass definition with a virtualization container runtime (in this example, Kata Containers combined with the Firecracker virtual machine monitor) that uses around 120MiB per Pod for the virtual machine and the guest OS:
```yaml # You need to change this example to match the actual runtime name, and per-Pod # resource overhead, that the container runtime is adding in your cluster. apiVersion: node.k8s.io/v1 kind: RuntimeClass metadata: name: kata-fc handler: kata-fc overhead: podFixed: memory: "120Mi" cpu: "250m" ```
Workloads which are created which specify the `kata-fc` RuntimeClass handler will take the memory and cpu overheads into account for resource quota calculations, node scheduling, as well as Pod cgroup sizing.
Consider running the given example workload, test-pod:
```yaml apiVersion: v1 kind: Pod metadata: name: test-pod spec: runtimeClassName: kata-fc containers: - name: busybox-ctr image: busybox:1.28 stdin: true tty: true resources: limits: cpu: 500m memory: 100Mi - name: nginx-ctr image: nginx resources: limits: cpu: 1500m memory: 100Mi ```
> Note: If only `limits` are specified in the pod definition, kubelet will deduce `requests` from those limits and set them to be the same as the defined `limits`.
At admission time the RuntimeClass [admission controller](/docs/reference/access-authn-authz/admission-controllers/) updates the workload's PodSpec to include the `overhead` as described in the RuntimeClass. If the PodSpec already has this field defined, the Pod will be rejected. In the given example, since only the RuntimeClass name is specified, the admission controller mutates the Pod to include an `overhead`.
After the RuntimeClass admission controller has made modifications, you can check the updated Pod overhead value:
```bash kubectl get pod test-pod -o jsonpath='{.spec.overhead}' ```
The output is:
``` map[cpu:250m memory:120Mi] ```
If a [ResourceQuota](/docs/concepts/policy/resource-quotas/) is defined, the sum of container requests as well as the `overhead` field are counted.
When the kube-scheduler is deciding which node should run a new Pod, the scheduler considers that Pod's `overhead` as well as the sum of container requests for that Pod. For this example, the scheduler adds the requests and the overhead, then looks for a node that has 2.25 CPU and 320 MiB of memory available.
Once a Pod is scheduled to a node, the kubelet on that node creates a new [cgroup](#gloss:cgroup) for the Pod. It is within this pod that the underlying container runtime will create containers.
If the resource has a limit defined for each container (Guaranteed QoS or Burstable QoS with limits defined), the kubelet will set an upper limit for the pod cgroup associated with that resource (cpu.cfs_quota_us for CPU and memory.limi …(trimmed)