Kubernetes/ConfigMap and Secrets

From Ever changing code
Jump to navigation Jump to search

ConfigMap

ConfigMap object allows to manage application's configuration using Kubernetes primitives. YAML below:

kubectl create configmap my-config-map --namespace=web -oyaml --dry-run > config-map.yml

<syntaxhighlightjs lang=yaml> apiVersion: v1 kind: ConfigMap metadata:

 creationTimestamp: null
 name: my-config-map
 namespace: web

data: # added when editing

 myKey: myValue1
 anotherKey: myValue2

</syntaxhighlightjs>

ConfigMap changes behaviours

There are four possible scenarios in response to ConfigMap changes, so you can choose your implementation:

1. ConfigMap that is mounted as a subPath volume has been modified.

  • The effect: the config file on the container won’t be updated.


2. ConfigMap had been modified and deployed in the cluster, then we deleted the pod manually.

  • The effect: the new pod will mount the updated version of the resource by itself.


3. ConfigMap has been modified; we used its hash sum in one of the Deployment annotations.

  • The effect: even though we updated the ConfigMap only, the Deployment has also changed. Therefore, the old pod will be automatically replaced with the new one containing the updated version of the resource. Note it will work only if you use Helm - Automatically Roll Deployments from Helm Tip&Tricks.


4. ConfigMap mounted as a directory has been modified.

  • The effect: the config file in the pod will be updated automatically, without restarting/rescheduling the pod.


ConfigMap mounted as a volume

ConfigMap mounted as subPath it won’t update until the pod restarts. The files are not-symlnks:

$ kubectl -n production exec go-conf-example-6b4cb86569-22vqv -- ls -lha /app/configfiles 
total 20K    
drwxr-xr-x    1 root     root        4.0K Mar  3 19:34 .
drwxr-xr-x    1 app      app         4.0K Mar  3 19:34 ..
-rw-r--r--    1 root     root          42 Mar  3 19:34 config.json
-rw-r--r--    1 root     root          47 Mar  3 19:34 database.yml


Mount it as a directory (without subPath), your container will get a continuously up-to-date config file. Notice in this case the files are actually the sym-links:

$ kubectl -n production exec go-conf-example-67c768c6fc-ccpwl -- ls -lha /app/configfiles 
total 12K    
drwxrwxrwx    3 root     root        4.0K Mar  3 19:40 .
drwxr-xr-x    1 app      app         4.0K Mar  3 19:34 ..
drwxr-xr-x    2 root     root        4.0K Mar  3 19:40 ..2020_03_03_16_40_36.675612011
lrwxrwxrwx    1 root     root          31 Mar  3 19:40 ..data -> ..2020_03_03_16_40_36.675612011
lrwxrwxrwx    1 root     root          18 Mar  3 19:40 config.json -> ..data/config.json
lrwxrwxrwx    1 root     root          19 Mar  3 19:40 database.yml -> ..data/database.yml

ConfigMap and Secrets mounted as a volume and $ENV

ConfigMap
ConfigMap value read into environment var ConfigMap mounted as a volume Secrets mounted as a volume
<syntaxhighlightjs lang=yaml>
  1. cat pod-configMap-env.yaml

apiVersion: v1 kind: Pod metadata:

 name: pod-configMap-env

spec:

 containers:
 - name: nginx
   image: nginx
   command: ['sh', '-c', "echo $(VAR) && sleep 600"]
   env:
   - name: VAR
     valueFrom:
       configMapKeyRef:
         name: kubeapp-config
         key: value1

</syntaxhighlightjs>

<syntaxhighlightjs lang=yaml>
  1. cat pod-configMap-volume.yaml

apiVersion: v1 kind: Pod metadata:

 name: pod-configMap-volume

spec:

 containers:
 - name: nginx
   image: nginx
   command: ['sh', '-c', "echo $(cat /etc/config/myKey && sleep 3600"]
   volumeMounts:
     - name: configmapvolume
       mountPath: /etc/config # this will be a directory
 volumes:
   - name: configmapvolume
     configMap:               # key will be a file name
       name: kube-configmap   # with value in content

</syntaxhighlightjs>

<syntaxhighlightjs lang=yaml>
  1. cat pod-secret-volume.yaml

apiVersion: v1 kind: Pod metadata:

 name: pod-secret-volume

spec:

 containers:
 - name: nginx
   image: nginx
   command: ['sh', '-c', "echo $(MY_VAR) && sleep 3600"]
   volumeMounts:
     - name: secretvolume
       mountPath: /etc/certs
 volumes:
   - name: secretvolume
     secret:
       secretName: kube-secret

</syntaxhighlightjs>


Deploy configMap

kubectl apply -f pod-configMap-env.yaml # pod-configMap-volume.yaml | pod-secret-volume.yaml
kubectl logs configmap-pod         #Get the logs from the pod displaying the value

configMap mounted as a volume with subPath

ConfigMap
Pod manifest ConfigMap
<syntaxhighlightjs lang=yaml>
  1. cat cm-configmap.yaml

apiVersion: v1 kind: ConfigMap metadata:

 name: cm-configmap

data:

 file1: |
   Line 1 in file1
 file2: |
   Line 1 in file2
   Line 2

</syntaxhighlightjs>

<syntaxhighlightjs lang=yaml>
  1. cat pod-configMap-volume.yaml

--- apiVersion: v1 kind: Pod metadata:

 name: centos-1
 namespace: pipaw

spec:

 containers:
 - name: centos
   image: centos:7.6.1810
   command:
     - sleep
     - "7200"
   volumeMounts:
   - name: configmap-volume
     mountPath: /etc/file1
     subPath: file1 # lookup on CM data, the filename is \
   - name: configmap-volume    # a data.address key here 'address'
     mountPath: /file2
     subPath: file2
   - name: configmap-volume
     mountPath: /etc/file2
     subPath: file2
 volumes:
 - name: configmap-volume
   configMap:
     name: cm-configmap

</syntaxhighlightjs>

Note: SubPaths are not automatically updated when a ConfigMap is modified. Changes to a ConfigMap will need to be a new deployment which would result in the pods being recreated with the updated ConfigMap content.




Another way to provide values from a ConfigMap is to mount as a container's volume. The keys you can see within the container

kubectl exec configmaps-volume-kube -- ls  /etc/config
kubectl exec configmaps-volume-kube -- cat /etc/config/key1

Secrets

Secrets types:

SecretType = "Opaque"                                 // Opaque (arbitrary data; default)
SecretType = "kubernetes.io/service-account-token"    // Kubernetes auth token
SecretType = "kubernetes.io/dockercfg"                // Docker registry auth
SecretType = "kubernetes.io/dockerconfigjson"         // Latest Docker registry auth


Create secrets

kubectl create secret generic user-creds --from-literal=pass=pass123 --from-literal=user=john --save-config  -oyaml --dry-run=true --type=Opaque > secrets.yaml

<syntaxhighlightjs lang=yaml> apiVersion: v1 kind: Secret metadata:

 creationTimestamp: null
 name: user-creds

data: # keys contain b64 encoded values

 pass: cGFzczEyMw==
 user: am9obg==

type: Opaque </syntaxhighlightjs>


Another secret. stringData: specifying non-binary secret data in string form. It is provided as a write-only convenience method. All keys and values are merged into the data field on write. <syntaxhighlightjs lang=yaml> apiVersion: v1 kind: Secret metadata:

 name: kube-secret

stringData: # literal string, keys' values will be b64 encoded on write

 cert: 1234abc
 key: ca.crt

</syntaxhighlightjs>


Describe secrets

kubectl describe secrets kube-secret
Name:         kube-secret
Namespace:    default
Labels:       <none>
Annotations:  
Type:         Opaque

Data
====
cert:  5 bytes
key:   5 bytes


Reference secrets in pod spec

kubectl create secret generic user-creds --from-literal=user=john --from-literal=password=pass123 --save-config -oyaml --type=Opaque  --dry-run=true
ConfigMap
As a environment Secrets mounted volume
<syntaxhighlightjs lang=yaml>

apiVersion: v1 kind: Pod metadata:

 name: busybox-with-secret-env

spec:

 containers:
 - name: busybox
   image: busybox
   command: ['sh', '-c', 'echo \"secret env variable VAR=$VAR\" && sleep 3600']
   env:
   - name: VAR
     valueFrom:
       secretKeyRef:
         name: user-creds
         key: password

</syntaxhighlightjs>


Verify

kubectl logs busybox-with-secret-env
"secret env variable VAR=pass123"

kubectl exec -it busybox-with-secret-env -- /bin/env | grep VAR
VAR=pass123
<syntaxhighlightjs lang=yaml>

apiVersion: v1 kind: Pod metadata:

 name: busybox-with-secret-volume-mounted

spec:

 containers:
 - name: busybox
   image: busybox
   command: ['sh', '-c', "echo \"Secret in the password file: $(cat /etc/user-creds/password)\" && sleep 3600"]
   volumeMounts:
     - name: secretvolume
       mountPath: /etc/user-creds # this will be a directory
       readOnly: true # optional
 volumes:
 - name: secretvolume
   secret:                    # key will be a file name
     secretName: user-creds   # with value in the content

</syntaxhighlightjs>

Verify

kubectl logs busybox-with-secret-volume-mounted
Secret in the password file: pass123

kubectl -n secrets exec -it busybox-with-secret-volume-mounted -- /bin/ls -la /etc/user-creds
total 4 # note symlinks, these cause issues if you think to do cat /etc/user-creds/*
drwxrwxrwt    3 root     root           120 Oct 20 20:08 .
drwxr-xr-x    1 root     root          4096 Oct 20 20:08 ..
drwxr-xr-x    2 root     root            80 Oct 20 20:08 ..2019_10_20_20_08_32.538523033
lrwxrwxrwx    1 root     root            31 Oct 20 20:08 ..data -> ..2019_10_20_20_08_32.538523033
lrwxrwxrwx    1 root     root            15 Oct 20 20:08 password -> ..data/password
lrwxrwxrwx    1 root     root            11 Oct 20 20:08 user -> ..data/user

kubectl exec busybox-with-secret-volume-mounted -- /bin/cat /etc/user-creds/{user,password}; echo
johnpass123

Patch a secret content

cat >cert-1.pfx <<-EOF
foo
EOF

k create secret generic cert.pfx --from-file=cert-1.pfx
k get    secret cert.pfx -ojson | jq -r '.data."cert-1.pfx"' | base64 -d
Piotr

cat >cert-2.pfx <<-EOF
bar
EOF

# Patch with base64
kubectl patch secret cert.pfx -p="{\"data\":{\"cert-1.pfx\": \"$(base64 cert-2.pfx)\"}}" -v=1
k get    secret cert.pfx -ojson | jq -r '.data."cert-1.pfx"' | base64 -d
bar

# Patch with clear text
kubectl patch secret cert.pfx -p="{\"stringData\":{\"cert-1.pfx\": \"No Base64 updade string\"}}" -v=1
k get    secret cert.pfx -ojson | jq -r '.data."cert-1.pfx"' | base64 -d
No Base64 updade string

configMap reload

  • Kubernetes ConfigMap Reload - trigger a reload when Kubernetes ConfigMaps are updated. It watches mounted volume dirs and notifies the target process that the config map has been changed. Used by prometheus as a sidecar

<syntaxhighlightjs lang="powershell"> kind: Pod apiVersion: v1 metadata:

 name: prometheus-788c945c9c-vpv4s
  1. ...

spec:

 volumes:
   - name: config-volume
     configMap:
       name: prometheus
       defaultMode: 420
  1. ...
 containers:
   - name: prometheus-server-configmap-reload
     image: 'jimmidyson/configmap-reload:v0.3.0' # latest v0.4.0
     args:
       - '--volume-dir=/etc/config'
       - '--webhook-url=http://127.0.0.1:9090/-/reload'
     resources: {}
     volumeMounts:
       - name: config-volume
         readOnly: true
         mountPath: /etc/config

</syntaxhighlightjs>

References