Network Traffic Shaping in Kubernetes: Topology Aware Routing

One of the challenges of having distributed systems in the cloud is keeping network traffic in the same availability zone.

Photo by Lars Kienle on Unsplash

An example scenario:

You have multiple nodes running in different zones in the cloud (e.g., eu-west-1a, eu-west-1b)

You have two applications: App1 and App2.

Each application has multiple containers, and the containers are running in different zones.

App1 frequently sends requests to App2.

How can you make sure that the containers of App1 make requests to the containers of App2, which are situated in the same zone?

To be clear, App1-Container1 sends the request to App2’s containers via the Service of App2.

This is how the apps are configured:

App1-Container1 is located in eu-west-1a
App1-Container2 is located in eu-west-1b

App2-Container1 is located in eu-west-1a
App2-Container2 is located in eu-west-1b

The Kubernetes Service component distributes traffic at random by design. This means that if App1-Container1 makes a request to App2, it can end up in a different availability zone.

Due to cross-zone communication, this could result in network slowness and extra expenses.

How can we prevent it?

Since version 1.21, Kubernetes has got a solution for this issue: Topology Aware Routing

I will deploy an example application:

00-app1-deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app1
spec:
  replicas: 2
  selector:
    matchLabels:
      app: app1
  template:
    metadata:
      labels:
        app: app1
    spec:
      containers:
        - image: ailhan/web-debug:zone
          name: app1

Apply:

➜  ~ kubectl apply -f 00-app1-deployment.yaml
deployment.apps/app1 created

01-app2-deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app2
spec:
  replicas: 2
  selector:
    matchLabels:
      app: app2
  template:
    metadata:
      labels:
        app: app2
    spec:
      containers:
        - image: ailhan/web-debug:zone
          name: app2
          ports:
            - containerPort: 80

Apply:

➜  ~ kubectl apply -f 01-app2-deployment.yaml
deployment.apps/app2 created

Create a service for App2:

02-app2-svc.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: app2-svc
spec:
  selector:
    app: app2
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

Here’s my setup:

App1 and App2 both have several containers operating in distinct availability zones.

The endpoint and endpoint slices setup is as follows:

Let’s try sending a request from App1 to App2:

The requests are spread at random among the various availability zones.

Let’s make our app2-svc Service topology aware

03-app2-svc.yaml

---
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.kubernetes.io/topology-mode: Auto
  name: app2-svc
spec:
  selector:
    app: app2
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

Apply:

➜  ~ kubectl apply -f 03-app2-svc.yaml
service/app2-svc configured

Let’s look at the service:

Let’s attempt to send App1 and App2 a request:

Topology Aware Routing ensures that requests from App1 to App2 remain within the same availability zone.

What happens if App2 is using a single container?

I deleted one of App2’s containers. All queries from App1 to App2 will be routed to a single App2 container:

P.S. : To keep things simple, I didn’t specify the code executing in App2. It retrieves the current zone via the Kubernetes API. To use the Kubernetes API, I created a Service Account, ClusterRole, and so on.