⎈ k8s knowledge compiler

Kubernetes API Server Bypass Risks [page]deterministic

Security architecture information relating to the API server and other components

concepts

The Kubernetes API server is the main point of entry to a cluster for external parties (users and services) interacting with it.

As part of this role, the API server has several key built-in security controls, such as audit logging and [admission controllers](#gloss:admission-controller). However, there are ways to modify the configuration or content of the cluster that bypass these controls.

This page describes the ways in which the security controls built into the Kubernetes API server can be bypassed, so that cluster operators and security architects can ensure that these bypasses are appropriately restricted.

## Static Pods {#static-pods}

The [kubelet](#gloss:kubelet) on each node loads and directly manages any manifests that are stored in a named directory or fetched from a specific URL as [*static Pods*](/docs/tasks/configure-pod-container/static-pod) in your cluster. The API server doesn't manage these static Pods. An attacker with write access to this location could modify the configuration of static pods loaded from that source, or could introduce new static Pods.

Static Pods are restricted from accessing other objects in the Kubernetes API. For example, you can't configure a static Pod to mount a Secret from the cluster. However, these Pods can take other security sensitive actions, such as using `hostPath` mounts from the underlying node.

By default, the kubelet creates a [mirror pod](#gloss:mirror-pod) so that the static Pods are visible in the Kubernetes API. However, if the attacker uses an invalid namespace name when creating the Pod, it will not be visible in the Kubernetes API and can only be discovered by tooling that has access to the affected host(s).

If a static Pod fails admission control, the kubelet won't register the Pod with the API server. However, the Pod still runs on the node. For more information, refer to [kubeadm issue #1541](https://github.com/kubernetes/kubeadm/issues/1541#issuecomment-487331701).

### Mitigations {#static-pods-mitigations}

  • Only [enable the kubelet static Pod manifest functionality](/docs/tasks/configure-pod-container/static-pod/#static-pod-creation) if required by the node.
  • If a node uses the static Pod functionality, restrict filesystem access to the static Pod manifest directory or URL to users who need the access.
  • Restrict access to kubelet configuration parameters and files to prevent an attacker setting a static Pod path or URL.
  • Regularly audit and centrally report all access to directories or web storage locations that host static Pod manifests and kubelet configuration files.

## The kubelet API {#kubelet-api}

The kubelet provides an HTTP API that is typically exposed on TCP port 10250 on cluster worker nodes. The API might also be exposed on control plane nodes depending on the Kubernetes distribution in use. Direct access to the API allows for disclosure of information about the pods running on a node, the logs from those pods, and execution of commands in every container running on the node.

Some of these endpoints support Websocket protocols via HTTP `GET` requests, which are authorized with the get verb. This means that get permission on `nodes/proxy` is not a read-only permission, and authorizes access to endpoints which can be used to execute commands in any container running on the node.

When Kubernetes cluster users have RBAC access to `Node` object sub-resources, that access serves as authorization to interact with the kubelet API. The exact access depends on which sub-resource access has been granted, as detailed in [kubelet authorization](/docs/reference/access-authn-authz/kubelet-authn-authz/#kubelet-authorization).

Direct access to the kubelet API is not subject to admission control and is not logged by Kubernetes audit logging. An attacker with direct access to this API may be able to bypass controls that detect or prevent certain actions.

The kubelet API can be configured to authenticate requests in a number of …(trimmed)

Sources

concepts/security/api-server-bypass-risks.md · docKubernetes API Server Bypass Risks

Related (11)

references Admission Controlleradmission controllers conf=1
references Kubeletkubelet conf=1
references Mirror Podmirror pod conf=1
part_of Static Pods {#static-pods}describes conf=1
part_of The kubelet API {#kubelet-api}describes conf=1
part_of The etcd APIdescribes conf=1
part_of Container runtime socket {#runtime-socket}describes conf=1
part_of Mitigations {#static-pods-mitigations}describes conf=1
part_of Mitigationsdescribes conf=1
part_of Mitigations {#etcd-api-mitigations}describes conf=1
part_of Mitigations {#runtime-socket-mitigations}describes conf=1

← all Docs