Kubernetes/Security and RBAC
API Server and Role Base Access Control
Once the API server has determined who you are (whether a pod or a user), the authorization is handled by RBAC.
To prevent unauthorized users from modifying the cluster state, RBAC is used by defining roles and role bindings for a user. A service account resource is created for a pod to determine what control has over the cluster state. For example, the default service account will not allow you to list the services in a namespace.
The Kubernetes API server provides CRUD actions (Create, Read, Update, Delete) interface for interacting with cluster state over a RESTful API. API calls can come only from 2 sources:
- kubectl
- POD
There is 4 stage process
- Authentication
- Authorization
- Admission
- Writing the configuration state CRUD actions to persistent store etcd database
Example plugins:
- serviceaccount plugin applies default serviceaccount to pods that don't explicitly specify
RBAC is managed by 4 resources, divided over 2 groups
Group-1 namespace resources | Group-2 cluster level resources | resources type |
---|---|---|
roles | cluster roles | defines what can be done |
role bindings | cluster role bindings | defines who can do it |
When deploying a pod a default serviceaccount is assigned if not specified in the pod manifest. The serviceaccount represents an identity of an app running on a pod. Token file holds authentication token. Let's create a namespace and create a test pod to try to list available services.
kubectl create ns rbac kubectl run apitest --image=nginx -n rbac #create test container, to run API call test from
Each pod has serviceaccount, the API authentication token is on a pod. When a pod makes API call uses the token, this allows to assumes the serviceaccount, so it gets identity. You can preview the token on the pod.
kubectl -n rbac1 exec -it apitest-<UID> -- /bin/sh #connect to the container shell #display token and namespace that allows to connect to API server from this pod root$ cat /var/run/secrets/kubernetes.io/serviceaccount/{token,namespace} #call API server to list K8s services in 'rbac' namespace root$ curl localhost:8001/api/v1/namespaces/rbac/services
List all serviceaccounts. Serviceaccounts can only be used within the same namespace.
kubectl get serviceaccounts -n rbac kubectl get secrets NAME TYPE DATA AGE default-token-qqzc7 kubernetes.io/service-account-token 3 39h kubectl get secrets default-token-qqzc7 -o yaml #display secrets
ServiceAccount
The API server is first evaluating if the request is coming from a service account or a normal user /or normal user account meeting, a private key, a user store or even a file with a list of user names and passwords. Kubernetes doesn't have objects that represent normal user accounts, and normal users cannot be added to the cluster through.
kubectl get serviceaccounts #or 'sa' in short kubectl create serviceaccount jenkins kubectl get serviceaccounts jenkins -o yaml apiVersion: v1 kind: ServiceAccount metadata: creationTimestamp: "2019-08-05T07:10:40Z" name: jenkins namespace: default resourceVersion: "678" selfLink: /api/v1/namespaces/default/serviceaccounts/jenkins uid: 21cba4bb-b750-11e9-86b3-0800274143a9 secrets: - name: jenkins-token-cspjm kubectl get secret [secret_name]
Assign ServiceAccoubt to a pod
apiVersion: v1 kind: Pod metadata: name: busybox namespace: default spec: serviceAccountName: jenkins #<-- ServiceAccount containers: - image: busybox:1.28.4 command: - sleep - "3600" imagePullPolicy: IfNotPresent name: busybox restartPolicy: Always #Verify kubectl.exe get pods -o yaml | sls serviceAccount {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"busybox","namespace":"default"},"spec":{"c ontainers":[{"command":["sleep","3600"],"image":"busybox:1.28.4","imagePullPolicy":"IfNotPresent","name":"busybox"}],"r estartPolicy":"Always","serviceAccountName":"jenkins"}} - mountPath: /var/run/secrets/kubernetes.io/serviceaccount serviceAccount: jenkins serviceAccountName: jenkins
Create Administrative account
This is a process of setting up a new remote administrator.
kubectl.exe config set-credentials piotr --username=piotr --password=password #new section in ~/.kube/config has been added: users: - name: user1 ... - name: piotr user: password: password username: piotr #create clusterrolebinding, this is for authonomus users not-recommended kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous created #copy server ca.crt laptop$ scp ubuntu@k8s-cluster.acme.com:/etc/kubernetes/pki/ca.crt . #set kubeconfig kubectl config set-cluster kubernetes --server=https://k8s-cluster.acme.com:6443 --certificate-authority=ca.crt --embed-certs=true #Create context kubectl config set-context kubernetes --cluster=kubernetes --user=piotr --namespace=default #Use contect to current kubectl config use-context kubernetes
Create a role (namespaced permissions)
The role describes what actions can be performed. This role allows to list services from a web namespace.
apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: namespace: web #this need to be created beforehand name: service-reader rules: - apiGroups: [""] verbs: ["get", "list"] resources: ["services"]
The role does not specify who can do it. Thus we create a roleBinding with a user, serviceAccount or group. The roleBinding can only reference a single role, but can bind to multi: users, serviceAccounts or groups
kubectl create rolebinding roleBinding-test --role=service-reader --serviceaccount=web:default -n web # Verify access has been granted curl localhost:8001/api/v1/namespaces/web/services
Create a clusterrole (cluster-wide permissions)
# Create a ClusterRole to access PersistentVolumes: kubectl create clusterrole pv-reader --verb=get,list --resource=persistentvolumes # Create a ClusterRoleBinding for the cluster role: kubectl create clusterrolebinding pv-test --clusterrole=pv-reader --serviceaccount=web:default
The YAML for a pod that includes a curl and proxy container:
apiVersion: v1 kind: Pod metadata: name: curlpod namespace: web spec: containers: - image: tutum/curl command: ["sleep", "9999999"] name: main - image: linuxacademycontent/kubectl-proxy name: proxy restartPolicy: Always
Create the pod that will allow you to curl directly from the container:
kubectl apply -f curl-pod.yaml # Get the pods in the web namespace: kubectl get pods -n web # Open a shell to the container: kubectl exec -it curlpod -n web -- sh #Access PersistentVolumes (cluster-level) from the pod: curl localhost:8001/api/v1/persistentvolumes
List all API resources
PS C:> kubectl.exe proxy Starting to serve on 127.0.0.1:8001