Setting Default Limits in a Kubernetes Namespace: LimitRange

adil
6 min readAug 1, 2023

One of the challenges in the containerization world is to use system resources efficiently. You can limit the system resources per pod/container.

Photo by Joshua Hoehne on Unsplash

Here’s how you can set memory/CPU limits:

apiVersion: apps/v1
kind: Deployment
metadata:
name: blog-test-deployment
spec:
replicas: 1
selector:
matchLabels:
app: blog-test
template:
metadata:
name: blog-test-pod
labels:
app: blog-test
spec:
containers:
- name: blog-test-container
image: ubuntu
command: ["/bin/bash", "-c", "--"]
args: ["while true; do sleep 3000; done;"]
resources:
requests:
memory: 24Mi
cpu: 125m
limits:
memory: 128Mi
cpu: 500m

This approach has two drawbacks:

  1. If you forget to add a limit, the pod will try to consume all system resources.
  2. The person who deploys this YAML file may replace the values with higher values and consume more resources than they should.

There should be a higher-order method to limit system resources per pod in a namespace. There is one: LimitRange

LimitRange

LimitRange allows you to set limitations per namespace.

Let’s say you want to have a namespace called development . The pods in this namespace should consume no more than 1 GB of memory and 1 CPU core.

01-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
name: development

Create the namespace:

➜  ~ kubectl apply -f 01-namespace.yaml
namespace/development created

02-limitrange.yaml

apiVersion: v1
kind: LimitRange
metadata:
name: blog-limitrange
namespace: development
spec:
limits:
- max:
cpu: 1000m
memory: 1Gi
type: Pod

1000m in Kubernetes signifies 1 CPU core or 1 vCPU on the cloud.

Create:

➜  ~ kubectl create -f 02-limitrange.yml
limitrange/blog-limitrange created
➜ ~ kubectl describe limitranges -n development
Name: blog-limitrange
Namespace: development
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Pod memory - 1Gi - - -
Pod cpu - 1 - - -

Try deploying a Pod with two CPU Cores:

03-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: cpu-problem-nginx
namespace: development
spec:
containers:
- image: nginx
name: cpu-problem-nginx
resources:
requests:
cpu: 2000m
limits:
cpu: 2000m

Create:

➜  ~ kubectl create -f 03-pod.yaml
Error from server (Forbidden): error when creating "03-pod.yaml": pods "cpu-problem-ubuntu" is forbidden: [maximum cpu usage per Pod is 1, but limit is 2, maximum memory usage per Pod is 1Gi. No limit is specified]

The pod creation request was denied due to CPU limits.

Try deploying a Pod with 2 Gigabytes of memory:

04-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: memory-problem-nginx
namespace: development
spec:
containers:
- image: nginx
name: memory-problem-nginx
resources:
requests:
memory: 2Gi
limits:
memory: 2Gi

Create:

➜  ~ kubectl create -f 04-pod.yaml
Error from server (Forbidden): error when creating "04-pod.yaml": pods "memory-problem-nginx" is forbidden: [maximum cpu usage per Pod is 1. No limit is specified, maximum memory usage per Pod is 1Gi, but limit is 2147483648]

Please notice that there are now two errors:

  1. The maximum CPU core was never mentioned in the 04-pod.yaml file. As a result, this pod will attempt to utilise all available CPUs. However, in the Limit Range, we set 1 CPU Core.
  2. The memory-problem-nginx pod is attempting to allocate 2 GiB of Memory which is in excess of the memory limit.

Try deploying a Pod with 0.5 CPU cores and 2 GiB of memory:

05-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: memory-problem-nginx
namespace: development
spec:
containers:
- image: nginx
name: memory-problem-nginx
resources:
requests:
cpu: 500m
memory: 2Gi
limits:
cpu: 500m
memory: 2Gi

Create:

➜  ~ kubectl create -f 05-pod.yaml
Error from server (Forbidden): error when creating "05-pod.yaml": pods "memory-problem-nginx" is forbidden: maximum memory usage per Pod is 1Gi, but limit is 2147483648

Please notice that the CPU error has been eliminated.

Try deploying a pod with one CPU core and one gigabyte of memory:

06-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: good-nginx
namespace: development
spec:
containers:
- image: nginx
name: good-nginx
resources:
requests:
cpu: 1000m
memory: 1Gi
limits:
cpu: 1000m
memory: 1Gi

Create:

➜  limits kubectl create -f 06-pod.yaml
pod/good-nginx created
➜ limits kubectl get pods -n development
NAME READY STATUS RESTARTS AGE
good-nginx 1/1 Running 0 6s

It has been successfully deployed.

Multiple Containers in One Pod

Please notice that the limit range type Pod was set in the 02-limitrange.yaml file.

Let’s try deploying multiple containers, each with 1 CPU Core and 1 GiB of memory:

07-multi-containers-one-pods.yaml

apiVersion: v1
kind: Pod
metadata:
name: bad-nginx-redis
namespace: development
spec:
containers:
- image: nginx
name: bad-nginx
resources:
requests:
cpu: 1000m
memory: 1Gi
limits:
cpu: 1000m
memory: 1Gi
- image: redis
name: bad-redis
resources:
requests:
cpu: 1000m
memory: 1Gi
limits:
cpu: 1000m
memory: 1Gi

Create:

➜  ~ kubectl create -f 07-multi-containers-one-pod.yaml
Error from server (Forbidden): error when creating "07-multi-containers-one-pod.yaml": pods "bad-nginx-redis" is forbidden: [maximum memory usage per Pod is 1Gi, but limit is 2147483648, maximum cpu usage per Pod is 1, but limit is 2]

Kubernetes throws an error when each container wants to allocate 1 GiB of Memory and use 1 CPU Core. Because we have a 1 GiB of memory and 1 CPU core limit per Pod. We require two CPU cores and two gigabytes of memory in the 07-multi-containers-one-pod.yaml file.

08-multi-containers-one-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: good-nginx-redis
namespace: development
spec:
containers:
- image: nginx
name: good-nginx
resources:
requests:
cpu: 500m
memory: 500Mi
limits:
cpu: 500m
memory: 500Mi
- image: redis
name: good-redis
resources:
requests:
cpu: 500m
memory: 500Mi
limits:
cpu: 500m
memory: 500Mi

Create:

➜  ~ kubectl create -f 08-multi-containers-one-pod.yaml
pod/good-nginx-redis created
➜ ~ kubectl get pods -n development
NAME READY STATUS RESTARTS AGE
good-nginx-redis 2/2 Running 0 23s

Delete the limit range:

➜  ~ kubectl delete -f 02-limitrange.yml
limitrange "blog-limitrange" deleted

Define a limit range with Type: Container

09-limitrange.yaml

apiVersion: v1
kind: LimitRange
metadata:
name: blog-limitrange
namespace: development
spec:
limits:
- max:
cpu: 1000m
memory: 1Gi
type: Container

Create:

➜  ~ kubectl create -f 09-limitrange.yaml
limitrange/blog-limitrange created
➜ ~ kubectl describe limitranges -n development
Name: blog-limitrange
Namespace: development
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container cpu - 1 1 1 -
Container memory - 1Gi 1Gi 1Gi -

Let’s try to deploy 07-multi-containers-one-pod.yaml file again:

➜  ~ kubectl create -f 07-multi-containers-one-pod.yaml
pod/bad-nginx-redis created
➜ ~ kubectl get pods -n development
NAME READY STATUS RESTARTS AGE
bad-nginx-redis 2/2 Running 0 33s

It has been successfully deployed. Because the Limit type is Container, not Pod. Each container is limited to 1 GiB of Memory and 1 CPU Core.

Remove the current limit range:

➜  ~ kubectl delete limitranges blog-limitrange -n dev
limitrange "blog-limitrange" deleted

Assign default limits

It is possible to define default limitations for each container

10-limitrange.yaml

apiVersion: v1
kind: LimitRange
metadata:
name: blog-limitrange
namespace: development
spec:
limits:
- max:
cpu: 1000m
memory: 1Gi
min:
cpu: 100m
memory: 32Mi
default:
cpu: 500m
memory: 128Mi
defaultRequest:
cpu: 250m
memory: 64Mi
type: Container

Create:

➜  ~ kubectl create -f 10-limitrange.yaml
limitrange/blog-limitrange created
➜ ~ kubectl describe limitranges -n development
Name: blog-limitrange
Namespace: development
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container cpu 100m 1 250m 500m -
Container memory 32Mi 1Gi 64Mi 128Mi -

Let’s deploy a pod without specifying any limits:

11-pod.yaml

apiVersion: v1
kind: Pod
metadata:
name: default-nginx
namespace: development
spec:
containers:
- image: nginx
name: default-nginx

Create:

➜  ~ kubectl create -f 11-pod.yaml
pod/default-nginx created
➜ ~ kubectl describe pod/default-nginx -n development
Name: default-nginx
Namespace: development
...
Limits:
cpu: 500m
memory: 128Mi
Requests:
cpu: 250m
memory: 64Mi
...

As we have previously configured default limits for the development namespace, Kubernetes sets restrictions for new pods.

P.S.: When using Type Pod, you cannot specify default or defaultRequest

--

--