Actions

Kubernetes/ConfigMap and Secrets

From Ever changing code

< Kubernetes

ConfigMap

ConfigMap object allows to manage application's configuration using Kubernetes primitives. YAML below:

kubectl create configmap my-config-map --namespace=web -oyaml --dry-run > config-map.yml
apiVersion: v1
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: my-config-map
  namespace: web
data:   # added when editing
  myKey: myValue1
  anotherKey: myValue2

ConfigMap changes behaviours

There are four possible scenarios in response to ConfigMap changes, so you can choose your implementation:

1. ConfigMap that is mounted as a subPath volume has been modified.

  • The effect: the config file on the container won’t be updated.


2. ConfigMap had been modified and deployed in the cluster, then we deleted the pod manually.

  • The effect: the new pod will mount the updated version of the resource by itself.


3. ConfigMap has been modified; we used its hash sum in one of the Deployment annotations.

  • The effect: even though we updated the ConfigMap only, the Deployment has also changed. Therefore, the old pod will be automatically replaced with the new one containing the updated version of the resource. Note it will work only if you use Helm - Automatically Roll Deployments from Helm Tip&Tricks.


4. ConfigMap mounted as a directory has been modified.

  • The effect: the config file in the pod will be updated automatically, without restarting/rescheduling the pod.


ConfigMap mounted as a volume

ConfigMap mounted as subPath it won’t update until the pod restarts. The files are not-symlnks:

$ kubectl -n production exec go-conf-example-6b4cb86569-22vqv -- ls -lha /app/configfiles 
total 20K    
drwxr-xr-x    1 root     root        4.0K Mar  3 19:34 .
drwxr-xr-x    1 app      app         4.0K Mar  3 19:34 ..
-rw-r--r--    1 root     root          42 Mar  3 19:34 config.json
-rw-r--r--    1 root     root          47 Mar  3 19:34 database.yml


Mount it as a directory (without subPath), your container will get a continuously up-to-date config file. Notice in this case the files are actually the sym-links:

$ kubectl -n production exec go-conf-example-67c768c6fc-ccpwl -- ls -lha /app/configfiles 
total 12K    
drwxrwxrwx    3 root     root        4.0K Mar  3 19:40 .
drwxr-xr-x    1 app      app         4.0K Mar  3 19:34 ..
drwxr-xr-x    2 root     root        4.0K Mar  3 19:40 ..2020_03_03_16_40_36.675612011
lrwxrwxrwx    1 root     root          31 Mar  3 19:40 ..data -> ..2020_03_03_16_40_36.675612011
lrwxrwxrwx    1 root     root          18 Mar  3 19:40 config.json -> ..data/config.json
lrwxrwxrwx    1 root     root          19 Mar  3 19:40 database.yml -> ..data/database.yml

ConfigMap and Secrets mounted as a volume and $ENV

ConfigMap
ConfigMap value read into environment var ConfigMap mounted as a volume Secrets mounted as a volume
# cat pod-configMap-env.yaml
apiVersion: v1
kind: Pod
metadata: 
  name: pod-configMap-env
spec:
  containers:
  - name: nginx
    image: nginx
    command: ['sh', '-c', "echo $(VAR) && sleep 600"]
    env:
    - name: VAR
      valueFrom:
        configMapKeyRef:
          name: kubeapp-config
          key: value1
# cat pod-configMap-volume.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-configMap-volume
spec:
  containers:
  - name: nginx
    image: nginx
    command: ['sh', '-c', "echo $(cat /etc/config/myKey && sleep 3600"]
    volumeMounts:
      - name: configmapvolume
        mountPath: /etc/config # this will be a directory
  volumes:
    - name: configmapvolume
      configMap:               # key will be a file name
        name: kube-configmap   # with value in content
# cat pod-secret-volume.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-secret-volume
spec:
  containers:
  - name: nginx
    image: nginx
    command: ['sh', '-c', "echo $(MY_VAR) && sleep 3600"]
    volumeMounts:
      - name: secretvolume
        mountPath: /etc/certs
  volumes:
    - name: secretvolume
      secret:
        secretName: kube-secret


Deploy configMap

kubectl apply -f pod-configMap-env.yaml # pod-configMap-volume.yaml | pod-secret-volume.yaml
kubectl logs configmap-pod         #Get the logs from the pod displaying the value

configMap mounted as a volume with subPath

ConfigMap
Pod manifest ConfigMap
# cat cm-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: cm-configmap
data:
  file1: |
    Line 1 in file1
  file2: |
    Line 1 in file2
    Line 2
# cat pod-configMap-volume.yaml
---
apiVersion: v1
kind: Pod
metadata:
  name: centos-1
  namespace: pipaw
spec:
  containers:
  - name: centos
    image: centos:7.6.1810
    command:
      - sleep
      - "7200"
    volumeMounts:
    - name: configmap-volume
      mountPath: /etc/file1
      subPath: file1 # lookup on CM data, the filename is \
    - name: configmap-volume    # a data.address key here 'address'
      mountPath: /file2
      subPath: file2
    - name: configmap-volume
      mountPath: /etc/file2
      subPath: file2
  volumes:
  - name: configmap-volume
    configMap:
      name: cm-configmap

Note: SubPaths are not automatically updated when a ConfigMap is modified. Changes to a ConfigMap will need to be a new deployment which would result in the pods being recreated with the updated ConfigMap content.




Another way to provide values from a ConfigMap is to mount as a container's volume. The keys you can see within the container

kubectl exec configmaps-volume-kube -- ls  /etc/config
kubectl exec configmaps-volume-kube -- cat /etc/config/key1

Secrets

Secrets types:

SecretType = "Opaque"                                 // Opaque (arbitrary data; default)
SecretType = "kubernetes.io/service-account-token"    // Kubernetes auth token
SecretType = "kubernetes.io/dockercfg"                // Docker registry auth
SecretType = "kubernetes.io/dockerconfigjson"         // Latest Docker registry auth


Create secrets

kubectl create secret generic user-creds --from-literal=pass=pass123 --from-literal=user=john --save-config  -oyaml --dry-run=true --type=Opaque > secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  creationTimestamp: null
  name: user-creds
data: # keys contain b64 encoded values
  pass: cGFzczEyMw==
  user: am9obg==
type: Opaque


Another secret. stringData: specifying non-binary secret data in string form. It is provided as a write-only convenience method. All keys and values are merged into the data field on write.

apiVersion: v1
kind: Secret
metadata:
  name: kube-secret
stringData: # literal string, keys' values will be b64 encoded on write
  cert: 1234abc
  key: ca.crt


Describe secrets

kubectl describe secrets kube-secret
Name:         kube-secret
Namespace:    default
Labels:       <none>
Annotations:  
Type:         Opaque

Data
====
cert:  5 bytes
key:   5 bytes


Reference secrets in pod spec

kubectl create secret generic user-creds --from-literal=user=john --from-literal=password=pass123 --save-config -oyaml --type=Opaque  --dry-run=true
ConfigMap
As a environment Secrets mounted volume
apiVersion: v1
kind: Pod
metadata: 
  name: busybox-with-secret-env
spec:
  containers:
  - name: busybox
    image: busybox
    command: ['sh', '-c', 'echo \"secret env variable VAR=$VAR\" && sleep 3600']
    env:
    - name: VAR
      valueFrom:
        secretKeyRef:
          name: user-creds
          key: password


Verify

kubectl logs busybox-with-secret-env
"secret env variable VAR=pass123"

kubectl exec -it busybox-with-secret-env -- /bin/env | grep VAR
VAR=pass123
apiVersion: v1
kind: Pod
metadata:
  name: busybox-with-secret-volume-mounted
spec:
  containers:
  - name: busybox
    image: busybox
    command: ['sh', '-c', "echo \"Secret in the password file: $(cat /etc/user-creds/password)\" && sleep 3600"]
    volumeMounts:
      - name: secretvolume
        mountPath: /etc/user-creds # this will be a directory
        readOnly: true # optional
  volumes:
  - name: secretvolume
    secret:                    # key will be a file name
      secretName: user-creds   # with value in the content

Verify

kubectl logs busybox-with-secret-volume-mounted
Secret in the password file: pass123

kubectl -n secrets exec -it busybox-with-secret-volume-mounted -- /bin/ls -la /etc/user-creds
total 4 # note symlinks, these cause issues if you think to do cat /etc/user-creds/*
drwxrwxrwt    3 root     root           120 Oct 20 20:08 .
drwxr-xr-x    1 root     root          4096 Oct 20 20:08 ..
drwxr-xr-x    2 root     root            80 Oct 20 20:08 ..2019_10_20_20_08_32.538523033
lrwxrwxrwx    1 root     root            31 Oct 20 20:08 ..data -> ..2019_10_20_20_08_32.538523033
lrwxrwxrwx    1 root     root            15 Oct 20 20:08 password -> ..data/password
lrwxrwxrwx    1 root     root            11 Oct 20 20:08 user -> ..data/user

kubectl exec busybox-with-secret-volume-mounted -- /bin/cat /etc/user-creds/{user,password}; echo
johnpass123

Patch a secret content

cat >cert-1.pfx <<-EOF
foo
EOF

k create secret generic cert.pfx --from-file=cert-1.pfx
k get    secret cert.pfx -ojson | jq -r '.data."cert-1.pfx"' | base64 -d
Piotr

cat >cert-2.pfx <<-EOF
bar
EOF

# Patch with base64
kubectl patch secret cert.pfx -p="{\"data\":{\"cert-1.pfx\": \"$(base64 cert-2.pfx)\"}}" -v=1
k get    secret cert.pfx -ojson | jq -r '.data."cert-1.pfx"' | base64 -d
bar

# Patch with clear text
kubectl patch secret cert.pfx -p="{\"stringData\":{\"cert-1.pfx\": \"No Base64 updade string\"}}" -v=1
k get    secret cert.pfx -ojson | jq -r '.data."cert-1.pfx"' | base64 -d
No Base64 updade string

configMap reload

  • Kubernetes ConfigMap Reload - trigger a reload when Kubernetes ConfigMaps are updated. It watches mounted volume dirs and notifies the target process that the config map has been changed. Used by prometheus as a sidecar
kind: Pod
apiVersion: v1
metadata:
  name: prometheus-788c945c9c-vpv4s
#...
spec:
  volumes:
    - name: config-volume
      configMap:
        name: prometheus
        defaultMode: 420
#...
  containers:
    - name: prometheus-server-configmap-reload
      image: 'jimmidyson/configmap-reload:v0.3.0' # latest v0.4.0
      args:
        - '--volume-dir=/etc/config'
        - '--webhook-url=http://127.0.0.1:9090/-/reload'
      resources: {}
      volumeMounts:
        - name: config-volume
          readOnly: true
          mountPath: /etc/config

References