Kubernetes/Helm

From Ever changing code
Jump to navigation Jump to search

Note: This content refers to helm3 as authored in April, 2020.

Install Helm

Note: Helm2 development is still active so, the latest tag may point actually to helm2 instead to helm3 

# latest version of helm2
LATEST=$(curl --silent "https://api.github.com/repos/helm/helm/releases/latest" | jq -r .tag_name); echo $LATEST

# latest version of helm3
LATEST=$(curl --silent "https://api.github.com/repos/helm/helm/releases" | jq -r .[].tag_name | sort -V | grep -v rc | tail -n -1); echo $LATEST
curl -LO https://get.helm.sh/helm-${LATEST}-linux-amd64.tar.gz
tar xzvf helm-${LATEST}-linux-amd64.tar.gz
sudo install linux-amd64/helm /usr/local/bin/helm
helm version


Helm-docs 

REPO=norwoodj/helm-docs 
LATEST=$(curl --silent "https://api.github.com/repos/${REPO}/releases" | jq -r .[].tag_name | sort -V | grep -v rc | tail -n -1 | tr -d 'v'); echo $LATEST
curl -LO https://github.com/norwoodj/helm-docs/releases/download/v${LATEST}/helm-docs_${LATEST}_Linux_x86_64.tar.gz
tar xzvf helm-docs_${LATEST}_Linux_x86_64.tar.gz
sudo install helm-docs /usr/local/bin/

Note: For bravehearts check this out https://github.com/roboll/helmfile, declarative way to install helm charts

Charts repository

Example below refers how to install metrics-server 

# Search repo
$ helm search hub metrics-server # searches 'hub' https://hub.helm.sh/
URL                                               	CHART VERSION	APP VERSION	DESCRIPTION                                       
https://hub.helm.sh/charts/bitnami/metrics-server 	4.2.0        	0.3.7      	Metrics Server...
https://hub.helm.sh/charts/stable/metrics-server  	2.11.1       	0.3.6      	Metrics Server...

# Add repo
$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm repo list 
NAME            	URL                                                   
gitlab          	https://charts.gitlab.io                              
bitnami         	https://charts.bitnami.com/bitnami 
$ helm search repo bitnami/metrics
NAME                  	CHART VERSION	APP VERSION	DESCRIPTION                                       
bitnami/metrics-server	4.2.0        	0.3.7      	Metrics Server is ...

# Install
helm install metrics-server bitnami/metrics-server -n metrics-server --dry-run # my-release is 'metrics-server'

# install 'metrics.k8s.io/v1beta1'
k get apiservices.apiregistration.k8s.io | grep metrics # if return nothing do below
helm upgrade --install metrics-server bitnami/metrics-server -n metrics-server --set apiService.create=true

# Test
k get apiservices.apiregistration.k8s.io | grep metrics
NAME                                   SERVICE                         AVAILABLE   AGE
v1beta1.metrics.k8s.io                 metrics-server/metrics-server   True        32s
kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes"
k top nodes

Diff plugin

Limitations and caveats: 

# Helm plugin install

## Option 1. Install with 'helm plugin'
helm plugin install https://github.com/databus23/helm-diff --version master # or tag ie v3.8.1

## Option 2.
VERSION=$(curl --silent "https://api.github.com/repos/databus23/helm-diff/releases/latest" | jq -r .tag_name); echo $VERSION
helm plugin install https://github.com/databus23/helm-diff/releases/download/$VERSION/helm-diff-linux-amd64.tgz

## Update with helm plugin
helm plugin list
helm plugin update diff

# Install using tarball, simply extract to '$HELM_PLUGINS'
eval $(helm env | grep PLUGINS) # eg. -> HELM_PLUGINS="/home/vagrant/.local/share/helm/plugins"
VERSION=$(curl --silent "https://api.github.com/repos/databus23/helm-diff/releases/latest" | jq -r .tag_name); echo $VERSION
wget https://github.com/databus23/helm-diff/releases/download/$VERSION/helm-diff-linux-amd64.tgz
tar -xzf helm-diff-linux-amd64.tgz -C $HELM_PLUGINS

# Usage
$> helm diff [command]

Available Commands:
  release     Shows diff between release manifests
  revision    Shows diff between revision manifests
  rollback    Show a diff explaining what a helm rollback could perform
  upgrade     Show a diff explaining what a helm upgrade would change.
  version     Show version of the helm diff plugin

# Examples
$> helm diff revision [flags] RELEASE REVISION1 REVISION2 --context 5
    # --context output NUM lines of context around changes (default -1)
$> helm diff upgrade RELEASE . -C 5
$> helm diff rollback RELEASE REVISION1 -C 5

# Compare the manifests details of a different releases created from the same chart
$> helm diff release RELEASE1 RELEASE2 -C 5

Operations

Helm command 

helm version --short # --> v3.1.3+g0a9a9a8
helm [command] -h    # get help create|env|get|history|install|lint|list|plugin|rollback|show|status|template|uninstall|upgrade
  # Flags:
  # --kube-context string -name of the kubeconfig context to use
  # --kubeconfig string -path to the kubeconfig file
  # -n, --namespace string


Operations 

# Add repo
helm repo add stable https://kubernetes-charts.storage.googleapis.com/
helm repo update                           # Make sure we get the latest list of charts

helm search repo stable/jenkins

# Set kubectl context
# Helm3 bug:does may not respect '--namespace' flag, so set default namespace
kubectl create ns jenkins
kubectl config set-context $(kubectl config current-context) --namespace=jenkins

# Install release
helm install [NAME]     [CHART]        [flags]
helm install jenkins-ci stable/jenkins                                    # release name: jenkins-ci
helm install jenkins-ci ./jenkins-2.192.1.tgz --set service.type=NodePort # install from a tarball package
helm install stable/mysql   --generate-name # release name will be generated

# Get information about releases
helm ls -A                                  # show a list of all deployed releases -A --all-namespaces
NAME       NAMESPACE REVISION UPDATED          STATUS   CHART          APP VERSION
jenkins-ci jenkins   1        2020-04-25 15:.. deployed jenkins-1.17.2 lts 

helm get all -n jenkins jenkins-ci # all|hooks|manifest|notes|values(user defined values)

# Uninstall release
helm uninstall jenkins-ci --namespace jenkins


Rollback
$> helm rollback <RELEASE> [REVISION] [flags]
$> helm rollback diy       2          --dry-run # <RELEASE> = diy
$> time helm rollback diy 2 --wait
    # --wait until all Pods, PVCs... are in a ready state before marking the release as successful. 
    # It will wait for as long as --timeout

# Note: Each rollback increments REVISION with +1
$> helm history diy
REVISION  UPDATED                   STATUS      CHART     APP VERSION DESCRIPTION
1         Mon Apr 27 10:39:38 2020  superseded  diy-0.1.0 6.4.2       Install complete
2         Mon Apr 27 12:24:54 2020  superseded  diy-0.1.0 6.4.2       Upgrade complete
3         Mon Apr 27 16:22:34 2020  superseded  diy-0.1.0 6.4.2       Upgrade complete
4         Mon Apr 27 16:35:31 2020  deployed    diy-0.1.0 6.4.2       Rollback to 2


Jenkins chart post install instructions or get notes commands
helm get notes -n jenkins jenkins-ci
NAME: jenkins-ci
LAST DEPLOYED: Sat Apr 25 15:33:36 2020
NAMESPACE: jenkins
STATUS: deployed
REVISION: 1
NOTES:
1. Get your 'admin' user password by running:
  printf $(kubectl get secret --namespace jenkins jenkins-ci -o jsonpath="{.data.jenkins-admin-password}" | base64 --decode);echo
2. Get the Jenkins URL to visit by running these commands in the same shell:
  export POD_NAME=$(kubectl get pods --namespace jenkins -l "app.kubernetes.io/component=jenkins-master" -l "app.kubernetes.io/instance=jenkins-ci" -o jsonpath="{.items[0].metadata.name}")
  echo http://127.0.0.1:8080
  kubectl --namespace jenkins port-forward $POD_NAME 8080:8080

3. Login with the password from step 1 and the username: admin


For more information on running Jenkins on Kubernetes, visit:
https://cloud.google.com/solutions/jenkins-on-container-engine

Subcharts

Subcharts and overriding variables

Subchart is a chart that lives in charts/ directory. It gets templated each time with the parent chart but can also be tested and deployed separately.

  • A subchart is considered "stand-alone", which means a subchart can never explicitly depend on its parent chart.
  • For that reason, a subchart cannot access the values of its parent.
  • A parent chart can override values for subcharts.
  • Helm has a concept of global values that can be accessed by all charts.


Overrides

If subchart has a variable variable1 this can be overriden from parent chart like below 

chartname:
  variable1: foo

Dependencies

Dependencies in Helm3 are driven by dependencies: block in Chart.yaml file. 

dependencies:
- name: nfs-server
  version: "0.1.0"
  condition: deployNfsServer.enabled  # (optional) A yaml path that resolves to a boolean, used for enabling/disabling charts (e.g. subchart1.enabled )
  repository: "file://../nfs-server"  # if 'repository' is unspecified assumes it exists in the charts directory
  enabled: true


At first we need to build the dependency package snapshot that will be used to build application. The snapshot will be packaged and placed in charts/ directory. Each time we do helm dependency update package get updated with any changes done to the referenced chart. 

# Update and build dependencies
helm dependency update # creates Chart.lock file
helm dependency build  # builds a package and places in charts/ directory eg. nfs-server-0.1.0.tgz

# --- step 1 - Chart.lock
cat Chart.lock
dependencies:
- name: nfs-server
  repository: file://../nfs-server
  version: 0.1.0
digest: sha256:e04f412832d56d97cfd153d2b2dd9eb5d6dad2c0fef65d3f92293672a2a6a043
generated: "2020-05-20T17:01:48.039203425Z"

# --- step 2 - package the dependency chart
#  | $ tree mft/charts/
#  | mft/charts/
#  | └── nfs-server-0.1.0.tgz

Now on each install the dependent chart will be installed along with the main chart. All manifests are applied in order:

  • aggregrated into a single set; then
  • sorted by type followed by name; and then
  • created/updated in that order.
helm upgrade --install mychart ./mychart --dry-run

Create a chart aka init

Note: Check The chart file structure as the helm create NAME does not include all optional files you may use 

helm version --short # -> v3.1.3+g0a9a9a8

helm create diy # diy = [NAME] # this creates standard file tree not all optional files are included
$> tree diy
diy
├── charts    # any charts upon which this chart depends
├── Chart.yaml
├── templates # no rigid naming pattern, recommended .yaml for YAML files and .tpl for helper
|   |         # all files here will be sent through the template engine
│   ├── deployment.yaml
│   ├── _helpers.tpl       # template helpers that you can re-use throughout the chart
│   ├── ingress.yaml
│   ├── NOTES.txt          # Optional. “help text” for your chart, displayed after installation
│   ├── serviceaccount.yaml
│   ├── service.yaml
│   └── tests
│       └── test-connection.yaml
└── values.yaml  # default configuration values for this chart

helm template diy | bat -l yaml --style=plain # display rendered yaml manifests

kubectl create ns diy
helm install ./diy --generate-name -n diy   # Generated name
  #                                       ____NAME______
  # podName       :                   pod/diy-1587926286-866d97bb75-vskns
  # serviceName   :               service/diy-1587926286
  # deploymentName: deployment.extensions/diy-1587926286
  # NAME                                          TYPE
  # secret/sh.helm.release.v1.diy-1587926286.v1   helm.sh/release.v1

helm install foo ./diy -n diy               # Custom name
  #                                       _NAME__
  # podName       :                   pod/foo-diy-6fd79d9bbd-nh5hl
  # serviceName   :               service/foo-diy
  # deploymentName: deployment.extensions/foo-diy
  # NAME                                          TYPE
  # secret/sh.helm.release.v1.foo.v1              helm.sh/release.v1

helm list -n diy
NAME           NAMESPACE REVISION UPDATED             STATUS   CHART     APP VERSION
diy-1587926286 diy       1        2020-04-26 19:38... deployed diy-0.1.0 1.16.0     
foo            diy       1        2020-04-26 20:02... deployed diy-0.1.0 1.16.0

Template

Everything in templates/ goes through templating engine before being rendered. Basic template might look like below: 

#            TEMPLATE                    --->            RENDER              --->            DEPLOYED
#                                                                             # helm install mychart ./mychart -n mychart
# cat templates/configmap.yaml             # helm template mychart/           # kubectl -n mychart get cm mychart-configmap -oyaml
| apiVersion: v1                           | apiVersion: v1                   | apiVersion: v1
| kind: ConfigMap                          | kind: ConfigMap                  | kind: ConfigMap
| metadata:                             -> | metadata:                    --> | metadata:
|   name: {{ .Release.Name }}-configmap    |   name: RELEASE-NAME-configmap   |   name: mychart-configmap
| data:                                    | data:                            | data:
|   myvalue: "Hello World Piotrek"         |   myvalue: "Hello World Piotrek" |   myvalue: Hello World Piotrek

Note: In kubectl get cm above creationTimestamp: and selfLink: data yaml keys have been not included for clarity of side-by-side comparison.


References

Text and spaces

Note: Details about whitespace control you can find in Flow Control in Controlling Whitespace section.


If an action's left delimiter (by default "{{") is followed immediately by a minus sign and ASCII space character ("{{- "), all trailing white space is trimmed from the immediately preceding text. Similarly, if the right delimiter ("}}") is preceded by a space and minus sign (" -}}"), all leading white space is trimmed from the immediately following text. In these trim markers, the ASCII space must be present; "{{-3}}" parses as an action containing the number -3.

For instance, when executing the template whose source is 

"{{23 -}} < {{- 45}}"
// the generated output would be

"23<45"

For this trimming, the definition of white space characters is the same as in Go: space, horizontal tab, carriage return, and newline.

Flow Control

Comments

# Inline, there is no spaces in between the `double-curly-brackets` and `forward-slash`
VariableFoo: {{ .Values.VariableFoo | quote }}{{/* FooComment __1 */}}

# Multiline_1
{{- /*
data:
  {{- $files := .Files }}
  {{- range tuple "files/sso.yaml" }}
  {{ . }}: |-
    {{ $files.Get . }}
  {{- end }}
*/ -}}

# Multiline_2, note this can cause some rendering issues sometimes
{{- /* stringData: - not base64 encoded
       data:       - base64 encoded, eg 'extra: YmFyCg==' */ -}}

Check if API Resource Exists

Using Capabilities.APIVerions.Has will allow you to check for the existence of a specific API version or resource. 

{{- if .Capabilities.APIVersions.Has "networking.k8s.io/v1beta1/Ingress" }}
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
...
{{- end }}

Package and push charts to OCI registry

Known workarounds

Invalid spec selector after upgrading helm template

This is caused by dynamic labels in selector as of Kubernetes version X field is immutable.


Steps to before upgrade without need of uninstalling charts: 

$ kubectl delete statefulsets.apps --cascade=false my-release

# Edit the deployment:
kubectl patch deployments my-release --type=json -p='[{"op": "remove", "path": "/spec/selector/matchLabels/chart"}]'

Variables and Spring functions

Variables are normally not "global". They are scoped to the block in which they are declared. 

{{- range $key, $val := .Values.favorite }}
{{ $key }}: {{ $val | quote }}
{{- end }}

variables $key and $val will only be in scope inside of the range block.


$ - variable that is always global, points to the root context. It's useful eg. when you are looping in a range and you need to know the chart's release name. 

{{- range .Values.tlsSecrets }}
apiVersion: v1
kind: Secret
metadata:
  name: {{ .name }}
  labels:
    # Many helm templates would use `.` below, but that will not work,
    # however `$` will work here
    app.kubernetes.io/name: {{ template "fullname" $ }}
    # I cannot reference .Chart.Name, but I can do $.Chart.Name
    helm.sh/chart: "{{ $.Chart.Name }}-{{ $.Chart.Version }}"
    ...
{{- end }}


Templates in helm cannot access variables. However, the context handed to the template is a dictionary. The spring library is accessible from within the Go templates and can be used.


Use function "set" to add a new key/value pair to a dictionary. 

$_ := set $myDict "name4" "value4"

Note that set returns the dictionary (a requirement of Go template functions), so you may need to trap the value as done above with the $_ assignment.

Resources

Retrieved from ‘http://wiki.ciscolinux.co.uk/index.php?title=Kubernetes/Helm&oldid=7003’