Streamline Java Application Deployment: Pack, Ship, and Unlock Distributed Tracing with Elastic APM on Kubernetes
In my last article, I dug into Distributed Tracing and exposed how to enable it in Java applications. We didn’t see yet how to deploy an application on Kubernetes and get distributed tracing insights. Several strategies can be considered, but the main point is how to minimize the impact of deploying APM agents on the whole delivery process.
To make it clearer, I will illustrate this setup by the following use case:
- We have an API “My wonderful API” which is instrumented through an Elastic APM agent.
- The data is then sent to the Elastic APM.
Now, if we dive into the “Wonderful System”, we can see the Wonderful Java application and the agent:
We can basically implement this architecture in two different ways:
- Deploying the agent in all of our Docker images
- Deploying the agent asides from the Docker images and using initContainers to bring the agent at the startup of our applications
We will then see how to lose couple application docker images to the apm agent one.
It could be really tempting to put the APM agents in the application’s Docker image.
Why? Because you just have to add the following lines of code in our Docker images definition:
Nonetheless, if you want to upgrade your agent, you will have to repackage it and redeploy all your Docker images.
For regular upgrades, it will not bother you, but, if you encounter a bug or a vulnerability, it will be tricky and annoying to do that.
What is why I prefer loose coupling the “business” applications Docker images to technical tools such as APM agents.
While looking around how to achieve this, I came across to the Kubernetes initContainers.
This kind of container is run only once during the startup of every pod. A bunch of commands is ran then on top of it. For our current use case, it will copy the javaagent into a volume such as an empty directory volume.
The main impact is to declare a volume in your Docker image:
It will be used by both the Docker container and the initContainer. We can consider it as a “bridge” between these two ones.
We also have to declare one environment variable:
Il will be used during the deployment to set up our Wonderful Java Application.
Now, let’s build our initContainer’s Docker image.
It is really straightforward. We can use for example, the following configuration:
We can now set up our Kubernetes Deployment to start the corresponding container and copy the Java agent.
Last but not least, we can now configure the pods where we run our Java applications.
We will use the
JAVA_OPTS environment variable to configure the location of the Java agent, and the Elastic APM Java system properties.
You can then configure your Kubernetes deployment as:
We have seen how to pack and deploy Distributed Tracing java agents and Java Applications built on top of Docker images. Obviously, my technical choice of using an InitContainer can be challenged regarding your technical context and how you are confortable with your delivery practices. You probably noticed I use an emptyDir to deploy the Java agent. Normally it will not be a big deal, but I advise you to check this usage with your Kubernetes SRE/Ops/Administrator first.
Anyway, I think it is worth it and the tradeoffs are more than acceptable because this approach are, in my opinion, more flexible than the first one.
Hope this helps!