What is a Kubernetes Ephemeral Container?

Best Way to Debug Containers

Kubernetes ephemeral container is one of the k8s beta features and here is how it works. When it comes to container security, distroless or minimal base images reduce the attack surface.

But the common concern in using a distroless or minimal image is that,

How do I take an exec session to troubleshoot if something goes wrong in the application? Because these images won’t even have a shell or any utilities required for troubleshooting.

Here is where ephemeral containers come in to picture.

An ephemeral container is a concept of adding a container in an exiting pod for debugging purposes. Let’s say you have a pod running on a minimal base image with just the application binaries and dependencies. Something went wrong, and you need to debug it.

Since it is a stripped-down minimal base image without a shell, you cannot perform a “kubectl exec” command. Here, you can add a debug container to an existing pod in real-time. This debug container would have all the required utilities to debug the application. (shell, curl, custom utilities, etc)

For example, let’s say you have a running pod named frontend, and you have an image with debug utilities called debug-image. The following command will add the debug-image container to the running frontend pod and take an exec session for debugging.

kubectl debug -it pods/frontend — image=debug-image

You can also debug a pod in CrashLoopBackOff state.

Note: Ephemeral Container was introduced in k8s v1.16 as an alpha feature, and now it is in beta as of 1.23. It is part of the Kubernetes core API.

Further reading:

[1] Introducing Ephemeral Containers

[2] K8s documentation - Ephemeral Containers