⎈ k8s knowledge compiler

Disruptions [page]deterministic

concepts

This guide is for application owners who want to build highly available applications, and thus need to understand what types of disruptions can happen to Pods.

It is also for cluster administrators who want to perform automated cluster actions, like upgrading and autoscaling clusters.

## Voluntary and involuntary disruptions

Pods do not disappear until someone (a person or a controller) destroys them, or there is an unavoidable hardware or system software error.

We call these unavoidable cases *involuntary disruptions* to an application. Examples are:

  • a hardware failure of the physical machine backing the node
  • cluster administrator deletes VM (instance) by mistake
  • cloud provider or hypervisor failure makes VM disappear
  • a kernel panic
  • the node disappears from the cluster due to cluster network partition
  • eviction of a pod due to the node being [out-of-resources](/docs/concepts/scheduling-eviction/node-pressure-eviction/).

Except for the out-of-resources condition, all these conditions should be familiar to most users; they are not specific to Kubernetes.

We call other cases *voluntary disruptions*. These include both actions initiated by the application owner and those initiated by a Cluster Administrator. Typical application owner actions include:

  • deleting the deployment or other controller that manages the pod
  • updating a deployment's pod template causing a restart
  • directly deleting a pod (e.g. by accident)

Cluster administrator actions include:

  • [Draining a node](/docs/tasks/administer-cluster/safely-drain-node/) for repair or upgrade.
  • Draining a node from a cluster to scale the cluster down (learn about [Node Autoscaling](/docs/concepts/cluster-administration/node-autoscaling/)).
  • Removing a pod from a node to permit something else to fit on that node.

These actions might be taken directly by the cluster administrator, or by automation run by the cluster administrator, or by your cluster hosting provider.

Ask your cluster administrator or consult your cloud provider or distribution documentation to determine if any sources of voluntary disruptions are enabled for your cluster. If none are enabled, you can skip creating Pod Disruption Budgets.

> Caution: Not all voluntary disruptions are constrained by Pod Disruption Budgets. For example, deleting deployments or pods bypasses Pod Disruption Budgets.

## Dealing with disruptions

Here are some ways to mitigate involuntary disruptions:

  • Ensure your pod [requests the resources](/docs/tasks/configure-pod-container/assign-memory-resource) it needs.
  • Replicate your application if you need higher availability. (Learn about running replicated [stateless](/docs/tasks/run-application/run-stateless-application-deployment/) and [stateful](/docs/tasks/run-application/run-replicated-stateful-application/) applications.)
  • For even higher availability when running replicated applications, spread applications across racks (using [anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)) or across zones (if using a [multi-zone cluster](/docs/setup/multiple-zones).)

The frequency of voluntary disruptions varies. On a basic Kubernetes cluster, there are no automated voluntary disruptions (only user-triggered ones). However, your cluster administrator or hosting provider may run some additional services which cause voluntary disruptions. For example, rolling out node software updates can cause voluntary disruptions. Also, some implementations of cluster (node) autoscaling may cause voluntary disruptions to defragment and compact nodes. Your cluster administrator or hosting provider should have documented what level of voluntary disruptions, if any, to expect. Certain configuration options, such as [using PriorityClasses](/docs/concepts/scheduling-eviction/pod-priority-preemption/) in your pod spec can also cause voluntary (and involuntary) disruptions.

## Pod disruption budgets

Kubernetes …(trimmed)

Sources

concepts/workloads/pods/disruptions.md · docDisruptions

Related (13)

references Disruptiondisruption conf=1
references Preemptionpreempted conf=1
references Tainttaint conf=1
references API-initiated evictioneviction using the Kubernetes API conf=1
references Node-pressure evictionnode pressure eviction conf=1
part_of Voluntary and involuntary disruptionsdescribes conf=1
part_of Dealing with disruptionsdescribes conf=1
part_of Pod disruption budgetsdescribes conf=1
part_of PodDisruptionBudget example {#pdb-example}describes conf=1
part_of {{% heading "whatsnext" %}}describes conf=1

← all Docs