Open in app

Sign in

Write

Sign in

Setting Default Limits in a Kubernetes Namespace: LimitRange

adil
6 min readAug 1, 2023

--

One of the challenges in the containerization world is to use system resources efficiently. You can limit the system resources per pod/container.

Photo by Joshua Hoehne on Unsplash

Here’s how you can set memory/CPU limits:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: blog-test-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: blog-test
  template:
    metadata:
      name: blog-test-pod
      labels:
        app: blog-test
    spec:
      containers:
        - name: blog-test-container
          image: ubuntu
          command: ["/bin/bash", "-c", "--"]
          args: ["while true; do sleep 3000; done;"]
          resources:
            requests:
              memory: 24Mi
              cpu: 125m
            limits:
              memory: 128Mi
              cpu: 500m

This approach has two drawbacks:

  1. If you forget to add a limit, the pod will try to consume all system resources.
  2. The person who deploys this YAML file may replace the values with higher values and consume more resources than they should.

There should be a higher-order method to limit system resources per pod in a namespace. There is one: LimitRange

LimitRange

LimitRange allows you to set limitations per namespace.

Let’s say you want to have a namespace called development . The pods in this namespace should consume no more than 1 GB of memory and 1 CPU core.

01-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: development

Create the namespace:

➜  ~ kubectl apply -f 01-namespace.yaml
namespace/development created

02-limitrange.yaml

apiVersion: v1
kind: LimitRange
metadata:
  name: blog-limitrange
  namespace: development
spec:
  limits:
  - max:
      cpu: 1000m
      memory: 1Gi
    type: Pod

1000m in Kubernetes signifies 1 CPU core or 1 vCPU on the cloud.

Create:

➜  ~ kubectl create -f 02-limitrange.yml
limitrange/blog-limitrange created
➜  ~ kubectl describe limitranges -n development
Name:       blog-limitrange
Namespace:  development
Type        Resource  Min  Max  Default Request  Default Limit  Max Limit/Request Ratio
----        --------  ---  ---  ---------------  -------------  -----------------------
Pod         memory    -    1Gi  -                -              -
Pod         cpu       -    1    -                -              -

Try deploying a Pod with two CPU Cores:

03-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: cpu-problem-nginx
  namespace: development
spec:
  containers:
  - image: nginx
    name: cpu-problem-nginx
    resources:
      requests:
        cpu: 2000m
      limits:
        cpu: 2000m

Create:

➜  ~ kubectl create -f 03-pod.yaml
Error from server (Forbidden): error when creating "03-pod.yaml": pods "cpu-problem-ubuntu" is forbidden: [maximum cpu usage per Pod is 1, but limit is 2, maximum memory usage per Pod is 1Gi.  No limit is specified]

The pod creation request was denied due to CPU limits.

Try deploying a Pod with 2 Gigabytes of memory:

04-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: memory-problem-nginx
  namespace: development
spec:
  containers:
  - image: nginx
    name: memory-problem-nginx
    resources:
      requests:
        memory: 2Gi
      limits:
        memory: 2Gi

Create:

➜  ~ kubectl create -f 04-pod.yaml
Error from server (Forbidden): error when creating "04-pod.yaml": pods "memory-problem-nginx" is forbidden: [maximum cpu usage per Pod is 1.  No limit is specified, maximum memory usage per Pod is 1Gi, but limit is 2147483648]

Please notice that there are now two errors:

  1. The maximum CPU core was never mentioned in the 04-pod.yaml file. As a result, this pod will attempt to utilise all available CPUs. However, in the Limit Range, we set 1 CPU Core.
  2. The memory-problem-nginx pod is attempting to allocate 2 GiB of Memory which is in excess of the memory limit.

Try deploying a Pod with 0.5 CPU cores and 2 GiB of memory:

05-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: memory-problem-nginx
  namespace: development
spec:
  containers:
  - image: nginx
    name: memory-problem-nginx
    resources:
      requests:
        cpu: 500m
        memory: 2Gi
      limits:
        cpu: 500m
        memory: 2Gi

Create:

➜  ~ kubectl create -f 05-pod.yaml
Error from server (Forbidden): error when creating "05-pod.yaml": pods "memory-problem-nginx" is forbidden: maximum memory usage per Pod is 1Gi, but limit is 2147483648

Please notice that the CPU error has been eliminated.

Try deploying a pod with one CPU core and one gigabyte of memory:

06-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: good-nginx
  namespace: development
spec:
  containers:
  - image: nginx
    name: good-nginx
    resources:
      requests:
        cpu: 1000m
        memory: 1Gi
      limits:
        cpu: 1000m
        memory: 1Gi

Create:

➜  limits kubectl create -f 06-pod.yaml
pod/good-nginx created
➜  limits kubectl get pods -n development
NAME                 READY   STATUS    RESTARTS   AGE
good-nginx           1/1     Running   0          6s

It has been successfully deployed.

Multiple Containers in One Pod

Please notice that the limit range type Pod was set in the 02-limitrange.yaml file.

Let’s try deploying multiple containers, each with 1 CPU Core and 1 GiB of memory:

07-multi-containers-one-pods.yaml

apiVersion: v1
kind: Pod
metadata:
  name: bad-nginx-redis
  namespace: development
spec:
  containers:
  - image: nginx
    name: bad-nginx
    resources:
      requests:
        cpu: 1000m
        memory: 1Gi
      limits:
        cpu: 1000m
        memory: 1Gi
  - image: redis
    name: bad-redis
    resources:
      requests:
        cpu: 1000m
        memory: 1Gi
      limits:
        cpu: 1000m
        memory: 1Gi

Create:

➜  ~ kubectl create -f 07-multi-containers-one-pod.yaml
Error from server (Forbidden): error when creating "07-multi-containers-one-pod.yaml": pods "bad-nginx-redis" is forbidden: [maximum memory usage per Pod is 1Gi, but limit is 2147483648, maximum cpu usage per Pod is 1, but limit is 2]

Kubernetes throws an error when each container wants to allocate 1 GiB of Memory and use 1 CPU Core. Because we have a 1 GiB of memory and 1 CPU core limit per Pod. We require two CPU cores and two gigabytes of memory in the 07-multi-containers-one-pod.yaml file.

08-multi-containers-one-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: good-nginx-redis
  namespace: development
spec:
  containers:
  - image: nginx
    name: good-nginx
    resources:
      requests:
        cpu: 500m
        memory: 500Mi
      limits:
        cpu: 500m
        memory: 500Mi
  - image: redis
    name: good-redis
    resources:
      requests:
        cpu: 500m
        memory: 500Mi
      limits:
        cpu: 500m
        memory: 500Mi

Create:

➜  ~ kubectl create -f 08-multi-containers-one-pod.yaml
pod/good-nginx-redis created
➜  ~ kubectl get pods -n development
NAME               READY   STATUS    RESTARTS   AGE
good-nginx-redis   2/2     Running   0          23s

Delete the limit range:

➜  ~ kubectl delete -f 02-limitrange.yml
limitrange "blog-limitrange" deleted

Define a limit range with Type: Container

09-limitrange.yaml

apiVersion: v1
kind: LimitRange
metadata:
  name: blog-limitrange
  namespace: development
spec:
  limits:
  - max:
      cpu: 1000m
      memory: 1Gi
    type: Container

Create:

➜  ~ kubectl create -f 09-limitrange.yaml
limitrange/blog-limitrange created
➜  ~ kubectl describe limitranges -n development
Name:       blog-limitrange
Namespace:  development
Type        Resource  Min  Max  Default Request  Default Limit  Max Limit/Request Ratio
----        --------  ---  ---  ---------------  -------------  -----------------------
Container   cpu       -    1    1                1              -
Container   memory    -    1Gi  1Gi              1Gi            -

Let’s try to deploy 07-multi-containers-one-pod.yaml file again:

➜  ~ kubectl create -f 07-multi-containers-one-pod.yaml
pod/bad-nginx-redis created
➜  ~ kubectl get pods -n development
NAME              READY   STATUS    RESTARTS   AGE
bad-nginx-redis   2/2     Running   0          33s

It has been successfully deployed. Because the Limit type is Container, not Pod. Each container is limited to 1 GiB of Memory and 1 CPU Core.

Remove the current limit range:

➜  ~ kubectl delete limitranges blog-limitrange -n dev
limitrange "blog-limitrange" deleted

Assign default limits

It is possible to define default limitations for each container

10-limitrange.yaml

apiVersion: v1
kind: LimitRange
metadata:
  name: blog-limitrange
  namespace: development
spec:
  limits:
  - max:
      cpu: 1000m
      memory: 1Gi
    min:
      cpu: 100m
      memory: 32Mi
    default:
      cpu: 500m
      memory: 128Mi
    defaultRequest:
      cpu: 250m
      memory: 64Mi
    type: Container

Create:

➜  ~ kubectl create -f 10-limitrange.yaml
limitrange/blog-limitrange created
➜  ~ kubectl describe limitranges -n development
Name:       blog-limitrange
Namespace:  development
Type        Resource  Min   Max  Default Request  Default Limit  Max Limit/Request Ratio
----        --------  ---   ---  ---------------  -------------  -----------------------
Container   cpu       100m  1    250m             500m           -
Container   memory    32Mi  1Gi  64Mi             128Mi          -

Let’s deploy a pod without specifying any limits:

11-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: default-nginx
  namespace: development
spec:
  containers:
  - image: nginx
    name: default-nginx

Create:

➜  ~ kubectl create -f 11-pod.yaml
pod/default-nginx created
➜  ~ kubectl describe pod/default-nginx -n development
Name:         default-nginx
Namespace:    development
...
    Limits:
      cpu:     500m
      memory:  128Mi
    Requests:
      cpu:        250m
      memory:     64Mi
...

As we have previously configured default limits for the development namespace, Kubernetes sets restrictions for new pods.

P.S.: When using Type Pod, you cannot specify default or defaultRequest

Kubernetes
Cloud
Cloud Native
K8s
Aws Eks

--

--

adil
adil

Written by adil

739 Followers
1 Following

https://www.linkedin.com/in/adililhan/

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams