Debug Services [page]deterministic

tasksnetworking

An issue that comes up rather frequently for new installations of Kubernetes is that a Service is not working properly. You've run your Pods through a Deployment (or other workload controller) and created a Service, but you get no response when you try to access it. This document will hopefully help you to figure out what's going wrong.

## Running commands in a Pod

For many steps here you will want to see what a Pod running in the cluster sees. The simplest way to do this is to run an interactive busybox Pod:

```none kubectl run -it --rm --restart=Never busybox --image=registry.k8s.io/busybox:1.27.2 sh ```

> Note: If you don't see a command prompt, try pressing enter.

If you already have a running Pod that you prefer to use, you can run a command in it using:

```shell kubectl exec <POD-NAME> -c <CONTAINER-NAME> -- <COMMAND> ```

## Setup

For the purposes of this walk-through, let's run some Pods. Since you're probably debugging your own Service you can substitute your own details, or you can follow along and get a second data point.

```shell kubectl create deployment hostnames --image=registry.k8s.io/serve_hostname ``` ```none deployment.apps/hostnames created ```

`kubectl` commands will print the type and name of the resource created or mutated, which can then be used in subsequent commands.

Let's scale the deployment to 3 replicas. ```shell kubectl scale deployment hostnames --replicas=3 ``` ```none deployment.apps/hostnames scaled ```

Note that this is the same as if you had started the Deployment with the following YAML:

```yaml apiVersion: apps/v1 kind: Deployment metadata: labels: app: hostnames name: hostnames spec: selector: matchLabels: app: hostnames replicas: 3 template: metadata: labels: app: hostnames spec: containers: - name: hostnames image: registry.k8s.io/serve_hostname ```

The label "app" is automatically set by `kubectl create deployment` to the name of the Deployment.

You can confirm your Pods are running:

```shell kubectl get pods -l app=hostnames ``` ```none NAME READY STATUS RESTARTS AGE hostnames-632524106-bbpiw 1/1 Running 0 2m hostnames-632524106-ly40y 1/1 Running 0 2m hostnames-632524106-tlaok 1/1 Running 0 2m ```

You can also confirm that your Pods are serving. You can get the list of Pod IP addresses and test them directly.

```shell kubectl get pods -l app=hostnames \ -o go-template='{{range .items}}{{.status.podIP}}{{"\n"}}{{end}}' ``` ```none 10.244.0.5 10.244.0.6 10.244.0.7 ```

The example container used for this walk-through serves its own hostname via HTTP on port 9376, but if you are debugging your own app, you'll want to use whatever port number your Pods are listening on.

From within a pod:

```shell for ep in 10.244.0.5:9376 10.244.0.6:9376 10.244.0.7:9376; do wget -qO- $ep done ```

This should produce something like:

``` hostnames-632524106-bbpiw hostnames-632524106-ly40y hostnames-632524106-tlaok ```

If you are not getting the responses you expect at this point, your Pods might not be healthy or might not be listening on the port you think they are. You might find `kubectl logs` to be useful for seeing what is happening, or perhaps you need to `kubectl exec` directly into your Pods and debug from there.

Assuming everything has gone to plan so far, you can start to investigate why your Service doesn't work.

## Does the Service exist?

The astute reader will have noticed that you did not actually create a Service yet - that is intentional. This is a step that sometimes gets forgotten, and is the first thing to check.

What would happen if you tried to access a non-existent Service? If you have another Pod that consumes this Service by name you would get something like:

```shell wget -O- hostnames ``` ```none Resolving hostnames (hostnames)... failed: Name or service not known. wget: unable …(trimmed)