Kubernetes/minikube

Minikube features

• LoadBalancer - using minikube tunnel
• Multi-cluster minikube start --profile cluster-1
• Multi-node cluster minikube start --profile cluster-2 --nodes 2
• NodePorts - using minikube service
• Persistent Volumes
• Ingress
• RBAC
• Dashboard - minikube dashboard
• Container runtimes - start --container-runtime
• Configure apiserver and kubelet options via command-line flags

Developer friendly features:

• Filesystem mounts
• Addons - a marketplace for developers to share configurations for running services on minikube
• NVIDIA GPU support - for machine learning

Release notes CHANGELOG.mb

Documentation minikube.sigs.k8s.io

Addons

List addons

$ minikube addons list --profile minikube-v1.14.9
|-----------------------------|----------------------|--------------|
|         ADDON NAME          |        PROFILE       |    STATUS    |
|-----------------------------|----------------------|--------------|
| dashboard                   | minikube-v1.14.9     | disabled     |
| default-storageclass        | minikube-v1.14.9     | enabled ✅   |
| efk                         | minikube-v1.14.9     | disabled     |
| freshpod                    | minikube-v1.14.9     | disabled     |
| gvisor                      | minikube-v1.14.9     | disabled     |
| helm-tiller                 | minikube-v1.14.9     | disabled     |
| ingress                     | minikube-v1.14.9     | enabled ✅   |
| ingress-dns                 | minikube-v1.14.9     | enabled ✅   |
| istio                       | minikube-v1.14.9     | disabled     |
| istio-provisioner           | minikube-v1.14.9     | disabled     |
| logviewer                   | minikube-v1.14.9     | disabled     |
| metrics-server              | minikube-v1.14.9     | disabled     |
| nvidia-driver-installer     | minikube-v1.14.9     | disabled     |
| nvidia-gpu-device-plugin    | minikube-v1.14.9     | disabled     |
| registry                    | minikube-v1.14.9     | disabled     |
| registry-aliases            | minikube-v1.14.9     | disabled     |
| registry-creds              | minikube-v1.14.9     | disabled     |
| storage-provisioner         | minikube-v1.14.9     | enabled ✅   |
| storage-provisioner-gluster | minikube-v1.14.9     | disabled     |
|-----------------------------|----------------------|--------------|

Usage

minikube -p efk start  --memory 8192
minikube -p efk addons enable efk     # pulling images and setting up can take ~10 min
minikube -p efk addons open efk       # opens Kibana

minikube addons enable heapster       # get some insight into CPU

# Show images
minikube addons images metrics-server
  - metrics-server has following images:
|---------------|--------------------------------------------------------------------------------------------------------------|------------------|
|  IMAGE NAME   |                                                DEFAULT IMAGE                                                 | DEFAULT REGISTRY |
|---------------|--------------------------------------------------------------------------------------------------------------|------------------|
| MetricsServer | metrics-server/metrics-server:v0.4.2@sha256:dbc33d7d35d2a9cc5ab402005aa7a0d13be6192f3550c7d42cba8d2d5e3a5d62 | k8s.gcr.io       |
|---------------|--------------------------------------------------------------------------------------------------------------|------------------|

Install or upgrade on Linux

# Latest from GitHub
VERSION=$(curl --silent "https://api.github.com/repos/kubernetes/minikube/releases/latest" | jq -r .tag_name); echo $VERSION
curl -LO https://github.com/kubernetes/minikube/releases/download/$VERSION/minikube-linux-amd64 \
      && sudo install minikube-linux-amd64 /usr/local/bin/minikube

# Latest from GoogleAPI
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 \
      && sudo install minikube-linux-amd64 /usr/local/bin/minikube

# Specific version from GitHub
curl -LO https://github.com/kubernetes/minikube/releases/download/v1.7.3/minikube-linux-amd64 \
      && sudo install minikube-linux-amd64 /usr/local/bin/minikube

$> $ minikube version 
minikube version: v1.17.0

Start on Windows

Install <syntaxhighlightjs lang="powershell"> choco install minikube kubernetes-cli </syntaxhighlightjs>

HyperV <syntaxhighlightjs lang="powershell">

1. HyperV set config (option 1)

minikube config set vm-driver hyperv minikube config set hyperv-virtual-switch "Default Switch" minikube start

1. HyperV single command (option 2)

minikube start --vm-driver hyperv --hyperv-virtual-switch k8s-node-1 --profile k8s-node-1

1. --hyperv-virtual-switch :- need to exist

</syntaxhighlightjs>

Virtualbox <syntaxhighlightjs lang="powershell"> minikube start --profile k8s-node-1

• minikube v1.2.0 on windows (amd64)
• Creating virtualbox VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...
• Configuring environment for Kubernetes v1.15.0 on Docker 18.09.6
• Downloading kubelet v1.15.0
• Downloading kubeadm v1.15.0
• Pulling images ...
• Launching Kubernetes ...
• Verifying: apiserver proxy etcd scheduler controller dns
• Done! kubectl is now configured to use "k8s-node-1"

PS C:\Users\Sylwia> kubectl.exe get nodes -owide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME minikube Ready master 23m v1.15.0 10.0.2.15 <none> Buildroot 2018.05.3 4.15.0 docker://18.9.6 </syntaxhighlightjs>

Start - Linux (Ubuntu)

Start sequence

VERSION=v1.18.9
time minikube start --kubernetes-version $VERSION --profile minikube-$VERSION
😄  [minikube-v1.18.9] minikube v1.19.0 on Ubuntu 20.04
✨  Automatically selected the docker driver. Other choices: virtualbox, ssh, none
👍  Starting control plane node minikube-v1.18.9 in cluster minikube-v1.18.9
🚜  Pulling base image ...
    > gcr.io/k8s-minikube/kicbase...: 357.67 MiB / 357.67 MiB  100.00% 4.70 MiB
🔥  Creating docker container (CPUs=2, Memory=3900MB) ...
🐳  Preparing Kubernetes v1.18.9 on Docker 20.10.5 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube-v1.18.9" cluster and "default" namespace by default

real	2m4.966s
user	0m6.658s
sys	0m3.768s


$> minikube profile list
|-----------------------------|-----------|---------|--------------|------|---------|---------|-------|
|           Profile           | VM Driver | Runtime |      IP      | Port | Version | Status  | Nodes |
|-----------------------------|-----------|---------|--------------|------|---------|---------|-------|
| minikube-v1.18.9            | docker    | docker  | 192.168.58.2 | 8443 | v1.18.9 | Running |     1 |
|-----------------------------|-----------|---------|--------------|------|---------|---------|-------|

$> minikube status -p minikube-v1.18.9
minikube-v1.18.9
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured

$> kubectl get nodes -o wide
NAME               STATUS   ROLES    AGE     VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
minikube-v1.18.9   Ready    master   2m34s   v1.18.9   192.168.58.2   <none>        Ubuntu 20.04.1 LTS   5.8.0-50-generic   docker://20.10.5


$ kubectl get all -o wide --all-namespaces
kubectl get all -o wide --all-namespaces
NAMESPACE     NAME                                                 READY   STATUS    RESTARTS   AGE     IP             NODE                     NOMINATED NODE   READINESS GATES
kube-system   pod/coredns-66bff467f8-wzjx7                         1/1     Running   0          4m59s   172.17.0.2     minikube-v1.18.9         <none>           <none>
kube-system   pod/etcd-minikube-v1.18.9                            1/1     Running   0          5m7s    192.168.49.2   minikube-v1.18.9         <none>           <none>
kube-system   pod/kube-apiserver-minikube-v1.18.9                  1/1     Running   0          5m7s    192.168.49.2   minikube-v1.18.9         <none>           <none>
kube-system   pod/kube-controller-manager-minikube-v1.18.9         1/1     Running   0          5m7s    192.168.49.2   minikube-v1.18.9         <none>           <none>
kube-system   pod/kube-proxy-zdqc9                                 1/1     Running   0          4m59s   192.168.49.2   minikube-v1.18.9         <none>           <none>
kube-system   pod/kube-scheduler-minikube-v1.18.9                  1/1     Running   0          5m7s    192.168.49.2   minikube-v1.18.9         <none>           <none>
kube-system   pod/storage-provisioner                              1/1     Running   1          5m13s   192.168.49.2   minikube-v1.18.9         <none>           <none>

NAMESPACE     NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE     SELECTOR
default       service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP                  5m15s   <none>
kube-system   service/kube-dns     ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP,9153/TCP   5m14s   k8s-app=kube-dns

NAMESPACE     NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE     CONTAINERS   IMAGES                          SELECTOR
kube-system   daemonset.apps/kube-proxy   1         1         1       1            1           kubernetes.io/os=linux   5m14s   kube-proxy   k8s.gcr.io/kube-proxy:v1.18.9   k8s-app=kube-proxy

NAMESPACE     NAME                      READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS   IMAGES                     SELECTOR
kube-system   deployment.apps/coredns   1/1     1            1           5m14s   coredns      k8s.gcr.io/coredns:1.6.7   k8s-app=kube-dns

NAMESPACE     NAME                                 DESIRED   CURRENT   READY   AGE     CONTAINERS   IMAGES                     SELECTOR
kube-system   replicaset.apps/coredns-66bff467f8   1         1         1       4m59s   coredns      k8s.gcr.io/coredns:1.6.7   k8s-app=kube-dns,pod-template-hash=66bff467f8

Usage (VirtualBox)

time minikube start --kubernetes-version v1.15.0 --profile minikube-v1.15.0

Get port exposed to host via Virtualbox used by minikube to interact with a cluster

$ sudo ss -ltpn
State  Recv-Q  Send-Q    Local Address:Port    Peer Address:Port                                               
LISTEN 0       128       127.0.0.53%lo:53           0.0.0.0:*      users:(("systemd-resolve",pid=1009,fd=13))  
LISTEN 0       10            127.0.0.1:45111 #<--   0.0.0.0:*      users:(("VBoxHeadless",pid=26918,fd=21))    
LISTEN 0       5             127.0.0.1:631          0.0.0.0:*      users:(("cupsd",pid=27629,fd=7))            
LISTEN 0       5                 [::1]:631             [::]:*      users:(("cupsd",pid=27629,fd=6))

$ vboxmanage showvminfo k8s-v1.15.0 | 'grep port ='                            vv
NIC 1 Rule(0):   name = ssh, protocol = tcp, host ip = 127.0.0.1, host port = 45111, guest ip = , guest port = 22
                                                                               ^^

Access to minikube VM using ssh

minikube ssh	cat /etc/*rel*
$ minikube ssh _ _ _ _ ( ) ( ) ___ ___ (_) ___ (_)| |/') _ _ | |_ __ /' _ ` _ `\| |/' _ `\| || , < ( ) ( )| '_`\ /'__`\ | ( ) ( ) || || ( ) || || |\`\ | (_) || |_) )( ___/ (_) (_) (_)(_)(_) (_)(_)(_) (_)`\___/'(_,__/'`\____) $	cat /etc/*rel* NAME=Buildroot VERSION=2018.05.3 ID=buildroot VERSION_ID=2018.05.3 PRETTY_NAME="Buildroot 2018.05.3" NAME=Buildroot VERSION=2018.05.3 ID=buildroot VERSION_ID=2018.05.3 PRETTY_NAME="Buildroot 2018.05.3"

#access Kubernetes Dashboard within Minikube
minikube dashboard
🤔  Verifying dashboard health ...
🚀  Launching proxy ...
🤔  Verifying proxy health ...
🎉  Opening http://127.0.0.1:44835/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/ in your default browser...

# Use kubectl to interact with the local cluster [[Kubernetes/Tools#create_pod]]
kubectl run --generator=run-pod/v1         hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080 #correct
kubectl run --generator=deployment/apps.v1 hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080 #deprecated
kubectl run       ^^default^^              hello-minikube --image=k8s.gcr.io/echoserver:1.4 --port=8080 #deprecated

# Expose a service as a NodePort
kubectl expose pod        hello-minikube --type=NodePort
kubectl expose deployment hello-minikube --type=NodePort #when pod was created via deployment deprecated option

# Determine the NodePort of the service
kubectl get service <hello-minikube> --output='jsonpath="{.spec.ports[0].nodePort}"'

# curl the endpoint
minikube ssh
curl -s -H 'Cache-Control: no-cache' http://<svc-cluster-ip>:8080 #note 'watch won't work as curl or service proxy 

#minikube makes it easy to open this exposed endpoint in your browser
minikube service hello-minikube

minikube stop   #stop your local cluster
minikube delete #delete your local cluster

Profiles

List profiles

$> minikube profile list # minikube v1.9.2
|-------------------|-----------|---------|------------|------|----------|---------|
|           Profile | VM Driver | Runtime |     IP     | Port | Version  | Status  |
|-------------------|-----------|---------|------------|------|----------|---------|
| minikube-v1.15.11 | docker    | docker  | 172.17.0.2 | 8443 | v1.15.11 | Stopped |
|-------------------|-----------|---------|------------|------|----------|---------|

Set and switch in between profiles

Setting the minikube profile does automatically change Kubernetes kubectl config set-context context.

minikube profile minikube-v1.15.11    # set the profile 'minikube-v1.15.11'
✅  minikube profile was successfully set to minikube-v1.15.11

# Configure the persistent default profile
minikube config set profile minikube-v1.15.11
minikube config view 
- profile: minikube-v1.15.11

# Manually switch kubectl context - not needed when using 'minikube profile'
kubectl config set-context minikube
Context "minikube" modified.

Profile local config files

$> ls -1 ~/.minikube/profiles/ # list all profiles
minikube-v1.15.11
minikube

# Minikube show current profile
$> minikube profile
minikube-v1.15.11
$> cat ~/.minikube/config/config.json | jq -r .profile #alternative option

# Kubectl  show current context
$ kubectl config current-context
$ kubectl config get-contexts   #get-clusters
CURRENT   NAME                  CLUSTER             AUTHINFO           NAMESPACE
*         minikube-v1.15.11     minikube-v1.15.11   minikube-v1.15.11
          minikube              minikube            minikube

Clean up if needed

minikube delete -p <profile>
kubectl config delete-context <context-name>
kubectl config delete-context <delete-cluster>

Example of ~/.minikube/profiles/<minikube-profile>/config.json

jq .MachineConfig config.json

jq .KubernetesConfig config.json

{ "KubernetesVersion": "v1.13.7", "NodeIP": "192.168.99.102", "NodePort": 8443, "NodeName": "minikube", "APIServerName": "minikubeCA", "APIServerNames": null, "APIServerIPs": null, "DNSDomain": "cluster.local", "ContainerRuntime": "docker", "CRISocket": "", "NetworkPlugin": "", "FeatureGates": "", "ServiceCIDR": "10.96.0.0/12", "ImageRepository": "", "ExtraOptions": null, "ShouldLoadCachedImages": true, "EnableDefaultCNI": false }

{ "KeepContext": false, "MinikubeISO": "https://storage.googleapis.com/minikube/iso/minikube-v1.2.0.iso", "Memory": 2048, "CPUs": 2, "DiskSize": 20000, "VMDriver": "virtualbox", "ContainerRuntime": "docker", "HyperkitVpnKitSock": "", "HyperkitVSockPorts": [], "XhyveDiskDriver": "ahci-hd", "DockerEnv": null, "InsecureRegistry": null, "RegistryMirror": null, "HostOnlyCIDR": "192.168.99.1/24", "HypervVirtualSwitch": "", "KvmNetwork": "default", "DockerOpt": null, "DisableDriverMounts": false, "NFSShare": [], "NFSSharesRoot": "/nfsshares", "UUID": "", "GPU": false, "Hidden": false, "NoVTXCheck": false }

Setting default configuration

It's useful to set default configuration, eg. you always work with AWS EKS, so it comes at certain versioning we could set as a default if not explicitly specified.

List configurable options

$ minikube config 
config modifies minikube config files using subcommands like "minikube config set driver kvm"
Configurable fields: 

 * driver              * log_dir                     * ShowDriverDeprecationNotification
 * vm-driver           * kubernetes-version          * ShowBootstrapperDeprecationNotification
 * container-runtime   * iso-url                     * insecure-registry
 * feature-gates       * WantUpdateNotification      * hyperv-virtual-switch
 * v                   * ReminderWaitPeriodInHours   * disable-driver-mounts
 * cpus                * WantReportError             * cache
 * disk-size           * WantReportErrorPrompt       * embed-certs
 * host-only-cidr      * WantKubectlDownloadMsg      * native-ssh
 * memory              * WantNoneDriverWarning
                       * profile
                       * bootstrapper
Available Commands:
  get         Gets the value of PROPERTY_NAME from the minikube config file
  set         Sets an individual value in a minikube config file
  unset       unsets an individual value in a minikube config file
  view        Display values currently set in the minikube config file

Usage:
  minikube config SUBCOMMAND [flags] [options]

Set default version of Kubernetes

$ minikube config set kubernetes-version v1.14.9
$ minikube config view 
- kubernetes-version: v1.14.9

Loadbalancer - minikube tunnel

A LoadBalancer service is the standard way to expose a service to the internet. With this method, each service gets it’s own IP address. In minikube services of type LoadBalancer can be exposed via the minikube tunnel command. It will run until Ctrl-C is hit.

minikube tunnel runs as a separate daemon, creating a network route on the host to the service CIDR of the cluster using the cluster’s IP address as a gateway. The tunnel command exposes the external IP directly to any program running on the host operating system. The low-level commands used to manage routes are /sbin/ip,/sbin/route that optionally can be added to suedors file to make a life a little easier.

$ minikube profile list
|-----------------|------------|----------------|-----------|--------------------|
|     Profile     | VM Driver  |     NodeIP     | Node Port | Kubernetes Version |
|-----------------|------------|----------------|-----------|--------------------|
| efk             | virtualbox | 192.168.99.113 |      8443 | v1.16.0            |
|-----------------|------------|----------------|-----------|--------------------|

# Before
$ kubectl -n efk get svc # note <pending> EXTERNAL-IP
NAMESPACE NAME                  TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)       
efk       service/elasticsearch LoadBalancer   10.104.149.221  <pending>       9200:30826/TCP
efk       service/kibana        LoadBalancer   10.97.198.117   <pending>       5601:32540/TCP

$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         vodafone.connec 0.0.0.0         UG    600    0        0 wlp4s0
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 wlp4s0
192.168.1.0     0.0.0.0         255.255.255.0   U     600    0        0 wlp4s0
192.168.99.0    0.0.0.0         255.255.255.0   U     0      0        0 vboxnet2

# After - EXTERNAL-IP have been assigned from a pool of 10.96.0.0/12
NAMESPACE NAME                  TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)       
efk       service/elasticsearch LoadBalancer   10.104.149.221  10.104.149.221  9200:30826/TCP
efk       service/kibana        LoadBalancer   10.97.198.117   10.97.198.117   5601:32540/TCP

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         vodafone.connec 0.0.0.0         UG    600    0        0 wlp4s0
10.96.0.0       192.168.99.113  255.240.0.0     UG    0      0        0 vboxnet2 # <- new route to minikube NodeIP created 
link-local      0.0.0.0         255.255.0.0     U     1000   0        0 wlp4s0
192.168.1.0     0.0.0.0         255.255.255.0   U     600    0        0 wlp4s0
192.168.99.0    0.0.0.0         255.255.255.0   U     0      0        0 vboxnet2

$ minikube --profile efk tunnel
Status:
        machine: efk
        pid: 8721
        route: 10.96.0.0/12 -> 192.168.99.113 # route status
        minikube: Running
        services: [elasticsearch, kibana]     # exposed services
    errors: 
                minikube: no errors
                router: no errors
                loadbalancer emulator: no errors
# Verify
$ nc -zv 10.97.198.117 5601 # kibana
Connection to 10.97.198.117 5601 port [tcp/*] succeeded!
$ nc -zv 10.104.149.221 9200 # elasticsearch
Connection to 10.104.149.221 9200 port [tcp/*] succeeded!
$ curl http://10.97.198.117:5601/status

Clean up based on info ~/.minikube/tunnels.json when shuts unexpected

minikube tunnel --cleanup

Avoid to typing password for minikube tunnel, by allowing a group or user to execute ip, route commands without asking for password. Example for Ubuntu 18.04 can be seen below.

$ sudo visudo
...
# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL
%admin ALL=(ALL) NOPASSWD: /sbin/ip,/sbin/route  # <- add this line in this order
...

multi node cluster with kvm

This is now supported natively using --nodes flag:

minikube start --profile minikube-v1.15.11 --nodes=2  --kubernetes-version v1.15.11
minikube ssh   --profile minikube-v1.15.11 --node=m02
minikube       --profile minikube-v1.15.11   node add         # add new node
minikube       --profile minikube-v1.15.11   node delete m03  # delete

$ kubectl get nodes
NAME                    STATUS     ROLES    AGE     VERSION
minikube-v1.15.11       Ready      master   3h25m   v1.15.11
minikube-v1.15.11-m02   NotReady   <none>   3h24m   v1.15.11
minikube-v1.15.11-m03   NotReady   <none>   16m     v1.15.11 # <- add / remove

Nodes names have following naming convention

• node-1-name: minikube-v1.14.9
• node-2-name: minikube-v1.14.9-m02

ingress and local DNS (ingress-dns)

TODO:

• Setting up ExternalDNS for Headless Services
• minikube-ingress-dns gitlab
• ingress-minikube Kubernetes docs

Add required addons

minikube addons enable ingress --profile minikube-v1.14.9
🌟  The 'ingress' addon is enabled
minikube addons enable ingress-dns --profile minikube-v1.14.9
🌟  The 'ingress-dns' addon is enabled

Add minikube ip as a dns server

Ubuntu 18.04 LTS, configure resolvconf

sudo apt install resolvconf
sudo -p /etc/resolvconf/resolv.conf.d       # possibly created by installation package
sudo vi /etc/resolvconf/resolv.conf.d/base
minikube ip --profile minikube-v1.14.9
192.168.99.126

# Update resolverconf
sudo vi /etc/resolvconf/resolv.conf.d/base
search test
nameserver 192.168.99.126
timeout 5

# Apply changes
sudo resolvconf -u
systemctl disable --now resolvconf.service 

# Test, Add the test ingress
kubectl apply -f https://raw.githubusercontent.com/kubernetes/minikube/master/deploy/addons/ingress-dns/example/example.yaml

# Validate DNS queries are returning A records
nslookup hello-john.test $(minikube ip --profile minikube-v1.14.9)
Server:		192.168.99.126
Address:	192.168.99.126#53

Non-authoritative answer:
Name:	hello-john.test
Address: 192.168.99.126
Name:	hello-john.test
Address: 192.168.99.126 # <- this is ingress IP address

$ kubectl -n kube-system get ingresses.networking.k8s.io example-ingress 
NAME              HOSTS                             ADDRESS          PORTS   AGE
example-ingress   hello-john.test,hello-jane.test   192.168.99.126   80      102s

Note: .local is a reserved TLD, do not use as this is a reserved TLD for mDNS and bind9 DNS servers

Note: .localhost domains will not correctly resolve on chromium since it is used as a loopback address. Instead use .test, .example, or .invalid

References

• minikube Github
• CHANGELOG.md Github
• Networking.md Github