Kubernetes/ConfigMap and Secrets

ConfigMap

ConfigMap object allows to manage application's configuration using Kubernetes primitives. YAML below:

kubectl create configmap my-config-map --namespace=web -oyaml --dry-run > config-map.yml

<syntaxhighlightjs lang=yaml> apiVersion: v1 kind: ConfigMap metadata:

 creationTimestamp: null
 name: my-config-map
 namespace: web

data: # added when editing

 myKey: myValue1
 anotherKey: myValue2

</syntaxhighlightjs>

ConfigMap changes behaviours

There are four possible scenarios in response to ConfigMap changes, so you can choose your implementation:

1. ConfigMap that is mounted as a subPath volume has been modified.

• The effect: the config file on the container won’t be updated.

2. ConfigMap had been modified and deployed in the cluster, then we deleted the pod manually.

• The effect: the new pod will mount the updated version of the resource by itself.

3. ConfigMap has been modified; we used its hash sum in one of the Deployment annotations.

• The effect: even though we updated the ConfigMap only, the Deployment has also changed. Therefore, the old pod will be automatically replaced with the new one containing the updated version of the resource. Note it will work only if you use Helm - Automatically Roll Deployments from Helm Tip&Tricks.

4. ConfigMap mounted as a directory has been modified.

• The effect: the config file in the pod will be updated automatically, without restarting/rescheduling the pod.

ConfigMap mounted as a volume

ConfigMap mounted as subPath it won’t update until the pod restarts. The files are not-symlnks:

$ kubectl -n production exec go-conf-example-6b4cb86569-22vqv -- ls -lha /app/configfiles 
total 20K    
drwxr-xr-x    1 root     root        4.0K Mar  3 19:34 .
drwxr-xr-x    1 app      app         4.0K Mar  3 19:34 ..
-rw-r--r--    1 root     root          42 Mar  3 19:34 config.json
-rw-r--r--    1 root     root          47 Mar  3 19:34 database.yml

Mount it as a directory (without subPath), your container will get a continuously up-to-date config file. Notice in this case the files are actually the sym-links:

$ kubectl -n production exec go-conf-example-67c768c6fc-ccpwl -- ls -lha /app/configfiles 
total 12K    
drwxrwxrwx    3 root     root        4.0K Mar  3 19:40 .
drwxr-xr-x    1 app      app         4.0K Mar  3 19:34 ..
drwxr-xr-x    2 root     root        4.0K Mar  3 19:40 ..2020_03_03_16_40_36.675612011
lrwxrwxrwx    1 root     root          31 Mar  3 19:40 ..data -> ..2020_03_03_16_40_36.675612011
lrwxrwxrwx    1 root     root          18 Mar  3 19:40 config.json -> ..data/config.json
lrwxrwxrwx    1 root     root          19 Mar  3 19:40 database.yml -> ..data/database.yml

ConfigMap and Secrets mounted as a volume and $ENV

ConfigMap

ConfigMap value read into environment var	ConfigMap mounted as a volume	Secrets mounted as a volume
<syntaxhighlightjs lang=yaml> cat pod-configMap-env.yaml apiVersion: v1 kind: Pod metadata: name: pod-configMap-env spec: containers: - name: nginx image: nginx command: ['sh', '-c', "echo $(VAR) && sleep 600"] env: - name: VAR valueFrom: configMapKeyRef: name: kubeapp-config key: value1 </syntaxhighlightjs>	<syntaxhighlightjs lang=yaml> cat pod-configMap-volume.yaml apiVersion: v1 kind: Pod metadata: name: pod-configMap-volume spec: containers: - name: nginx image: nginx command: ['sh', '-c', "echo $(cat /etc/config/myKey && sleep 3600"] volumeMounts: - name: configmapvolume mountPath: /etc/config # this will be a directory volumes: - name: configmapvolume configMap: # key will be a file name name: kube-configmap # with value in content </syntaxhighlightjs>	<syntaxhighlightjs lang=yaml> cat pod-secret-volume.yaml apiVersion: v1 kind: Pod metadata: name: pod-secret-volume spec: containers: - name: nginx image: nginx command: ['sh', '-c', "echo $(MY_VAR) && sleep 3600"] volumeMounts: - name: secretvolume mountPath: /etc/certs volumes: - name: secretvolume secret: secretName: kube-secret </syntaxhighlightjs>

Deploy configMap

kubectl apply -f pod-configMap-env.yaml # pod-configMap-volume.yaml | pod-secret-volume.yaml
kubectl logs configmap-pod         #Get the logs from the pod displaying the value

configMap mounted as a volume with subPath

ConfigMap

Pod manifest

ConfigMap

<syntaxhighlightjs lang=yaml> cat cm-configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: cm-configmap data: file1: | Line 1 in file1 file2: | Line 1 in file2 Line 2 </syntaxhighlightjs>

<syntaxhighlightjs lang=yaml> cat pod-configMap-volume.yaml --- apiVersion: v1 kind: Pod metadata: name: centos-1 namespace: pipaw spec: containers: - name: centos image: centos:7.6.1810 command: - sleep - "7200" volumeMounts: - name: configmap-volume mountPath: /etc/file1 subPath: file1 # lookup on CM data, the filename is \ - name: configmap-volume # a data.address key here 'address' mountPath: /file2 subPath: file2 - name: configmap-volume mountPath: /etc/file2 subPath: file2 volumes: - name: configmap-volume configMap: name: cm-configmap </syntaxhighlightjs>

Note: SubPaths are not automatically updated when a ConfigMap is modified. Changes to a ConfigMap will need to be a new deployment which would result in the pods being recreated with the updated ConfigMap content.

Another way to provide values from a ConfigMap is to mount as a container's volume. The keys you can see within the container

kubectl exec configmaps-volume-kube -- ls  /etc/config
kubectl exec configmaps-volume-kube -- cat /etc/config/key1

Secrets

Secrets types:

SecretType = "Opaque"                                 // Opaque (arbitrary data; default)
SecretType = "kubernetes.io/service-account-token"    // Kubernetes auth token
SecretType = "kubernetes.io/dockercfg"                // Docker registry auth
SecretType = "kubernetes.io/dockerconfigjson"         // Latest Docker registry auth

Create secrets

kubectl create secret generic user-creds --from-literal=pass=pass123 --from-literal=user=john --save-config  -oyaml --dry-run=true --type=Opaque > secrets.yaml

<syntaxhighlightjs lang=yaml> apiVersion: v1 kind: Secret metadata:

 creationTimestamp: null
 name: user-creds

data: # keys contain b64 encoded values

 pass: cGFzczEyMw==
 user: am9obg==

type: Opaque </syntaxhighlightjs>

Another secret. stringData: specifying non-binary secret data in string form. It is provided as a write-only convenience method. All keys and values are merged into the data field on write. <syntaxhighlightjs lang=yaml> apiVersion: v1 kind: Secret metadata:

 name: kube-secret

stringData: # literal string, keys' values will be b64 encoded on write

 cert: 1234abc
 key: ca.crt

</syntaxhighlightjs>

Describe secrets

kubectl describe secrets kube-secret
Name:         kube-secret
Namespace:    default
Labels:       <none>
Annotations:  
Type:         Opaque

Data
====
cert:  5 bytes
key:   5 bytes

Reference secrets in pod spec

kubectl create secret generic user-creds --from-literal=user=john --from-literal=password=pass123 --save-config -oyaml --type=Opaque  --dry-run=true

ConfigMap

As a environment

Secrets mounted volume

<syntaxhighlightjs lang=yaml> apiVersion: v1 kind: Pod metadata: name: busybox-with-secret-env spec: containers: - name: busybox image: busybox command: ['sh', '-c', 'echo \"secret env variable VAR=$VAR\" && sleep 3600'] env: - name: VAR valueFrom: secretKeyRef: name: user-creds key: password </syntaxhighlightjs> Verify kubectl logs busybox-with-secret-env "secret env variable VAR=pass123" kubectl exec -it busybox-with-secret-env -- /bin/env | grep VAR VAR=pass123

<syntaxhighlightjs lang=yaml> apiVersion: v1 kind: Pod metadata: name: busybox-with-secret-volume-mounted spec: containers: - name: busybox image: busybox command: ['sh', '-c', "echo \"Secret in the password file: $(cat /etc/user-creds/password)\" && sleep 3600"] volumeMounts: - name: secretvolume mountPath: /etc/user-creds # this will be a directory readOnly: true # optional volumes: - name: secretvolume secret: # key will be a file name secretName: user-creds # with value in the content </syntaxhighlightjs> Verify kubectl logs busybox-with-secret-volume-mounted Secret in the password file: pass123 kubectl -n secrets exec -it busybox-with-secret-volume-mounted -- /bin/ls -la /etc/user-creds total 4 # note symlinks, these cause issues if you think to do cat /etc/user-creds/* drwxrwxrwt 3 root root 120 Oct 20 20:08 . drwxr-xr-x 1 root root 4096 Oct 20 20:08 .. drwxr-xr-x 2 root root 80 Oct 20 20:08 ..2019_10_20_20_08_32.538523033 lrwxrwxrwx 1 root root 31 Oct 20 20:08 ..data -> ..2019_10_20_20_08_32.538523033 lrwxrwxrwx 1 root root 15 Oct 20 20:08 password -> ..data/password lrwxrwxrwx 1 root root 11 Oct 20 20:08 user -> ..data/user kubectl exec busybox-with-secret-volume-mounted -- /bin/cat /etc/user-creds/{user,password}; echo johnpass123

Patch a secret content

cat >cert-1.pfx <<-EOF
foo
EOF

k create secret generic cert.pfx --from-file=cert-1.pfx
k get    secret cert.pfx -ojson | jq -r '.data."cert-1.pfx"' | base64 -d
Piotr

cat >cert-2.pfx <<-EOF
bar
EOF

# Patch with base64
kubectl patch secret cert.pfx -p="{\"data\":{\"cert-1.pfx\": \"$(base64 cert-2.pfx)\"}}" -v=1
k get    secret cert.pfx -ojson | jq -r '.data."cert-1.pfx"' | base64 -d
bar

# Patch with clear text
kubectl patch secret cert.pfx -p="{\"stringData\":{\"cert-1.pfx\": \"No Base64 updade string\"}}" -v=1
k get    secret cert.pfx -ojson | jq -r '.data."cert-1.pfx"' | base64 -d
No Base64 updade string

configMap reload

• Kubernetes ConfigMap Reload - trigger a reload when Kubernetes ConfigMaps are updated. It watches mounted volume dirs and notifies the target process that the config map has been changed. Used by prometheus as a sidecar

<syntaxhighlightjs lang="powershell"> kind: Pod apiVersion: v1 metadata:

 name: prometheus-788c945c9c-vpv4s

1. ...

spec:

 volumes:
   - name: config-volume
     configMap:
       name: prometheus
       defaultMode: 420

1. ...

 containers:
   - name: prometheus-server-configmap-reload
     image: 'jimmidyson/configmap-reload:v0.3.0' # latest v0.4.0
     args:
       - '--volume-dir=/etc/config'
       - '--webhook-url=http://127.0.0.1:9090/-/reload'
     resources: {}
     volumeMounts:
       - name: config-volume
         readOnly: true
         mountPath: /etc/config

</syntaxhighlightjs>

References

• configure-pod-configmap
• Secretes