Use Carefully on Kubernetes: Liveness Probe, Readiness Probe, or Startup Probe
Imagine you have a slow-starting web application (Spring Boot?).
Before sending traffic to the web application, you have some requirements:
- You want to make sure the application launches successfully (Startup probe)
- If the application is receiving high volumes of traffic and is unresponsive for a while you do not want to send traffic to the application (Readiness probe)
- If the application’s listening port is unreachable due to internal issues of the application, terminate and restart the app (Liveness probe)
If you enable Startup Probe and Readiness/Liveness probes in Kubernetes, the Startup Probe rule must be successful before you can start using Readiness/Liveness probes.
Tips for the Liveness Probe
In my opinion, the liveness probe is the most challenging health check method. Because if the liveness probe rule fails, the pod will be killed. Depending on the restart policy, the pod may reboot. Your pods may reboot repeatedly if you misconfigure the liveness probe.
As a best practice, if a problem occurs, the app should kill itself instead of waiting for it to be killed from the outside. Otherwise, the liveness probe rule may restart the pod due to a minor glitch.
One of the most common uses for the liveness probe is to restart the application for memory leaks or deadlocks in multi-threaded apps.
Tips for the Readiness Probe
If you need to expose your pod through a service, it’s wise to enable the readiness probe on your pod. Because if the readiness probe rule fails, the service will stop forwarding traffic to the pod. The pod will not be killed or restarted if the readiness probe fails.
The readiness probe is often used to verify the health of a web application.
Tips for the Startup Probe
It is highly recommended that you enable the startup probe rule if you want to use the liveness probe. The startup probe rule works like a liveness probe rule. If the startup probe rule fails, the pod will be killed. Depending on the restart policy, the pod may reboot.
Quote from the official Kubernetes documentation (source):
If your container usually starts in more than initialDelaySeconds + failureThreshold x periodSeconds, you should specify a startup probe that checks the same endpoint as the liveness probe.
You should then set its failureThreshold high enough to allow the container to start, without changing the default values of the liveness probe. This helps to protect against deadlocks.
Carefully Set the Restart Policy and Failure Threshold
There are three different values for the restart policy: Always, OnFailure, and Never. The default value is Always.
Due to incorrect health check configurations, your application may be shut down and restarted repeatedly.
The default value for failureThreshold is 3. Since the default value is very low, your pod may restart frequently. It is highly recommended to adjust the value for failureThreshold depending on your business requirements
