In the microservices world, it is not uncommon to provide app config for the services. In the Java and Spring ecosystems, there are multiple ways to pass the configuration data to the application, like providing command line arguments but more traditionally application.properties or application.yaml configuration files are used to set up applications.
On the contrary, most of the time we have multiple environments such as local, test, and prod, and all of them require a new file to store the properties files. This file is coupled with the code, so once a change is required the file must be changed and committed to the repository, which will require a new application build and deployment.
We can observe a pattern where configs vary across deployments, but code does not. Because of this, it is crucial that we separate the config from the code.
Using environment variables makes life easier as the values are no longer hard coded alongside the codebase. At this point, it seems that there are multiple options, so the question is – how can we handle them at scale? Let’s dive into what Kubernetes offers us and what we can achieve with Kubernetes configuration maps.
Decoupling configuration with ConfigMap
The whole point of an app’s configuration is to keep the config options that vary between environments, or change frequently, separate from the application’s source code. If we think of a Pod descriptor as the source code for our app, it’s clear that we should move the configuration out of the Pod description.
ConfigMap is a Kubernetes API object that contains key value pairs, making it handy for us to store configuration in it. They will be exposed to the containers and can be processed in various ways:
- Inside a container that has command and args as configuration parameters in the Pod definition
- Defined environment variables for a container in the Pod spec
- By adding a file in the read-only volume for the application to read
- As code written to run inside the Pod that uses the Kubernetes API to read a ConfigMap
There are multiple ways to create a ConfigMap. I will show you a couple of examples.
I have minikube, kubectl, and docker installed and started on my computer. They are required to run the commands below.
Creating a ConfigMap can be done via command line with the “kubectl create configmap” command. The input can be literal values (key value pair) or even a config file. Apart from the configmap command line option, you can also define them from a yaml file.
$ kubectl create configmap netguru-config --from-literal=SOME_EXAMPLE=29 $ kubectl create -f netguru-config-2.yaml
Where the content of the
netguru-config-2.yaml is the following:
apiVersion: v1 kind: ConfigMap metadata: name: netguru-config-2 data: GREETER_MESSAGE: "Hello from Kubernetes" RANDOM_ENTRY: "random" binaryData: ENCODED_VALUE: SGVsbG8gV29ybGQh
It might be worth taking a look at the fields that are being used in the yaml file:
- metadata – ConfigMap metadata holds information about the name of the ConfigMap that will be created
- data – data field contains key value pairs in a regular string format
- binaryData – used for binary data encoded as Base-64 strings
Both the data and binaryData fields are optional.
Let’s make sure that the ConfigMap exists:
$ kubectl get configmap
And we can view the complete ConfigMap and the ConfigMap data with the following commands:
$ kubectl get configmap netguru-config -o yaml $ kubectl get configmap netguru-config-2 -o yaml
The ConfigMap's data field contains all the values we provided when creating the ConfigMap.
How you create ConfigMaps does not really matter that much, the result will be the same. You will have a Kubernetes ConfigMap object that can be used by your apps, even by multiple Pods in the same namespace.
Now that we have an example of a ConfigMap in the Kubernetes cluster, we are able to use it.
At the Pod definition, in the container spec we can use the env and envFrom properties to define the ConfigMap mapping. For testing purposes, I will use a publicly available container image:
apiVersion: v1 kind: pod metadata: name: netguru-pod spec: containers: - name: netguru-container image: httpd ports: - containerPort: 8080 env: - name: SOME_EXAMPLE valueFrom: configMapKeyRef: key: SOME_EXAMPLE name: netguru-config envFrom: - prefix: CUSTOM_ configMapRef: name: netguru-config-2
With the env we are able to provide a single environment variable where the envFrom will convert all the values from the ConfigMap to env values. Here you can see that I used a prefix, but it is not mandatory. If amended, the environment variable will have the same key as in the ConfigMap.
At this point, when the Pods start running, the application will be able to use the environment variables that were created by the ConfigMap.
Let’s take a look inside to see if the environment variables are there:
$ kubectl exec netguru-pod env
To be complete, at least on the mentioned level, check out what happens when we have larger config files. In this case, Kubernetes is able to pass them to the Pods as an attached volume. You have to simply define the ConfigMap and where the mounted ConfigMap should go. In case you have more than one entry in the ConfigMap, it will result in multiple files under the directory.
The one thing to keep in mind is that by default, when mounting a volume to the Pod, if there is already something in that path then all the files will be hidden from that point.
For additional information, check out the official documentation.
You can now see how to use ConfigMaps in the example above, so let's take a look at Secrets.
Some words about Secrets
They are pretty much the same as ConfigMaps, they are both API objects inside Kubernetes and they are used in a similar way. There are a couple of key differences though:
- Secrets are the preferred way to pass confidential information to the containers, be it passwords or binary data, like private keys.
- By default, Secrets are not encrypted so extra steps must be taken in order to make them so. Since they are a different object inside Kubernetes, it is possible to put a more fine-grained access control on them and it is highly recommended to do so.
To get a better understanding about the usage of Secrets, I recommend checking out the official documentation.
Spring Cloud Kubernetes project
Configuring the projects this way has proven to be so useful that Spring introduced ways to make them work more easily.
With the spring-cloud-starter-kubernetes-config project, we are able to configure the application to fetch specific ConfigMaps or Secrets through the Kubernetes API and they are made automatically available in the application as if you had them next to the codebase as properties or yaml files. Of course, Spring can work with volume attached configs too. In that case, it will use the file system instead of the API.
Another cool benefit is that it supports hot reloading of the application context and beans once a change is detected in the ConfigMap.
Best of it all, is that it is quite easy to set up:
- Add the org.springframework.cloud:spring-cloud-starter-kubernetes-config and spring-cloud-starter-bootstrap dependencies to your project
- Create a bootstrap.yaml file and add the following entries:
The official documentation highlights all the features and steps to make it work.
Why is this good for us?
- By following the widely-accepted software development best practices, the frequently changing configurations are decoupled from the source code, which makes updating them far easier.
- We are no longer forced to maintain numerous configuration files alongside the codebase, which also makes the process less error prone.
- If we have confidential data, we can use Secrets that are managed in a secure way, making them less likely to get compromised.
- Spring has built-in features to help us use Kubernetes ConfigMaps and Secrets, making the transition to them more transparent.
We learned what problems we are facing with traditional Spring application properties, what the benefits are of environment variables, and how we can provide them to our applications that are running in a Kubernetes environment, using Kubernetes ConfigMaps and Secrets.