Topology Aware Routing [page]deterministic

Prior to Kubernetes 1.27, this feature was known as _Topology Aware Hints_.

_Topology Aware Routing_ adjusts routing behavior to prefer keeping traffic in the zone it originated from. In some cases this can help reduce costs or improve network performance.

## Motivation

Kubernetes clusters are increasingly deployed in multi-zone environments. _Topology Aware Routing_ provides a mechanism to help keep traffic within the zone it originated from. When calculating the endpoints for a [Service](#gloss:Service), the EndpointSlice controller considers the topology (region and zone) of each endpoint and populates the hints field to allocate it to a zone. Cluster components such as [kube-proxy](#gloss:kube-proxy) can then consume those hints, and use them to influence how the traffic is routed (favoring topologically closer endpoints).

## Enabling Topology Aware Routing

Prior to Kubernetes 1.27, this behavior was controlled using the `service.kubernetes.io/topology-aware-hints` annotation.

You can enable Topology Aware Routing for a Service by setting the `service.kubernetes.io/topology-mode` annotation to `Auto`. When there are enough endpoints available in each zone, Topology Hints will be populated on EndpointSlices to allocate individual endpoints to specific zones, resulting in traffic being routed closer to where it originated from.

## When it works best

This feature works best when:

### 1. Incoming traffic is evenly distributed

If a large proportion of traffic is originating from a single zone, that traffic could overload the subset of endpoints that have been allocated to that zone. This feature is not recommended when incoming traffic is expected to originate from a single zone.

### 2. The Service has 3 or more endpoints per zone {#three-or-more-endpoints-per-zone} In a three zone cluster, this means 9 or more endpoints. If there are fewer than 3 endpoints per zone, there is a high (≈50%) probability that the EndpointSlice controller will not be able to allocate endpoints evenly and instead will fall back to the default cluster-wide routing approach.

## How It Works

The "Auto" heuristic attempts to proportionally allocate a number of endpoints to each zone. Note that this heuristic works best for Services that have a significant number of endpoints.

### EndpointSlice controller {#implementation-control-plane}

The EndpointSlice controller is responsible for setting hints on EndpointSlices when this heuristic is enabled. The controller allocates a proportional amount of endpoints to each zone. This proportion is based on the [allocatable](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable) CPU cores for nodes running in that zone. For example, if one zone had 2 CPU cores and another zone only had 1 CPU core, the controller would allocate twice as many endpoints to the zone with 2 CPU cores.

The following example shows what an EndpointSlice looks like when hints have been populated:

```yaml apiVersion: discovery.k8s.io/v1 kind: EndpointSlice metadata: name: example-hints labels: kubernetes.io/service-name: example-svc addressType: IPv4 ports: - name: http protocol: TCP port: 80 endpoints: - addresses: - "10.1.2.3" conditions: ready: true hostname: pod-1 zone: zone-a hints: forZones: - name: "zone-a" ```

### kube-proxy {#implementation-kube-proxy}

The kube-proxy component filters the endpoints it routes to based on the hints set by the EndpointSlice controller. In most cases, this means that the kube-proxy is able to route traffic to endpoints in the same zone. Sometimes the controller allocates endpoints from a different zone to ensure more even distribution of endpoints between zones. This would result in some traffic being routed to other zones.

## Safeguards

The Kubernetes control plane and the kube-proxy on each node apply some safeguard rules before using Topology Aware Hints. If these don't check out, the kub …(trimmed)