目录
- 一.系统环境
- 二.前言
- 三.PodSecurityPolicy 简介
- 四.为什么需要 PodSecurityPolicy
- 五.给客户端授权
- 六.启用PodSecurityPolicy（PSP）
- 七.PSP规则之禁止创建特权用户pod
- 八.PSP规则之数据卷
- 九.PSP规则之指定使用宿主机网络
- 十.PSP规则之pod使用特定的UID运行
- 十一.PSP规则之指定hostport端口
- 十二.PSP规则实战1
- 十三.PSP规则实战2:创建deployment
- 十四.总结
一.系统环境
本文主要基于Kubernetes1.22.2和Linux操作系统Ubuntu 18.04。
| 服务器版本 | docker软件版本 | Kubernetes(k8s)集群版本 | CPU架构 |
|---|---|---|---|
| Ubuntu 18.04.5 LTS | Docker version 20.10.14 | v1.22.2 | x86_64 |
Kubernetes集群架构：k8scludes1作为master节点，k8scludes2，k8scludes3作为worker节点。
| 服务器 | 操作系统版本 | CPU架构 | 进程 | 功能描述 |
|---|---|---|---|---|
| k8scludes1/192.168.110.128 | Ubuntu 18.04.5 LTS | x86_64 | docker，kube-apiserver，etcd，kube-scheduler，kube-controller-manager，kubelet，kube-proxy，coredns，calico | k8s master节点 |
| k8scludes2/192.168.110.129 | Ubuntu 18.04.5 LTS | x86_64 | docker，kubelet，kube-proxy，calico | k8s worker节点 |
| k8scludes3/192.168.110.130 | Ubuntu 18.04.5 LTS | x86_64 | docker，kubelet，kube-proxy，calico | k8s worker节点 |
二.前言
随着容器化和微服务架构的普及，Kubernetes 已经成为企业中编排和管理容器的主要平台。然而，随着集群中运行的容器数量和种类的增加，确保容器的安全性变得越来越重要。PodSecurityPolicy（PSP）是 Kubernetes 用于限制 Pod 创建的一组规则，以确保集群的安全性。在本文中，我们将深入了解 PSP 的概念、重要性以及如何在 Kubernetes 集群中实施。
使用PodSecurityPolicy（PSP）的前提是已经有一套可以正常运行的Kubernetes集群，关于Kubernetes(k8s)集群的安装部署，可以查看博客《Ubuntu 安装部署Kubernetes(k8s)集群》https://www.cnblogs.com/renshengdezheli/p/17632858.html。
三.PodSecurityPolicy 简介
PodSecurityPolicy 是一个内置的准入控制器， 允许集群管理员控制 Pod 规约中涉及安全的敏感内容。
首先，在集群中创建一个或多个 PodSecurityPolicy 资源来定义 Pod 必须满足的要求。 然后，创建 RBAC 规则来决定为特定的 Pod 应用哪个 PodSecurityPolicy。 如果 Pod 满足其 PSP 的要求，则照常被允许进入集群。 在某些情况下，PSP 还可以修改 Pod 字段，有效地为这些字段创建新的默认值。 如果 Pod 不符合 PSP 要求，则被拒绝进入集群，并且无法运行。
简单说来，就是PodSecurityPolicy 可以设置pod里的进程只能以某个UID运行，或者设置pod只能使用某一类型存储，不能使用其他，比如只能使用nfs存储。
四.为什么需要 PodSecurityPolicy
在 Kubernetes 中，我们定义了 Deployment、StatefulSet 和 Service 等资源。 这些资源代表软件应用程序的构建块。Kubernetes 集群中的各种控制器根据这些资源做出反应， 创建更多的 Kubernetes 资源或配置一些软件或硬件来实现我们的目标。
在大多数 Kubernetes 集群中，由 RBAC（基于角色的访问控制）规则 控制对这些资源的访问。 list、get、create、edit 和 delete 是 RBAC 关心的 API 操作类型， 但 RBAC 不考虑其所控制的资源中加入了哪些设置。例如， Pod 几乎可以是任何东西，例如简单的网络服务器，或者是特权命令提示（提供对底层服务器节点和所有数据的完全访问权限）。 这对 RBAC 来说都是一样的：Pod 就是 Pod 而已。
要控制集群中定义的资源允许哪些类型的设置，除了 RBAC 之外，还需要准入控制。 从 Kubernetes 1.3 开始，内置 PodSecurityPolicy 一直被作为 Pod 安全相关字段的准入控制机制。 使用 PodSecurityPolicy，可以防止“创建 Pod”这个能力自动变成“每个集群节点上的 root 用户”， 并且无需部署额外的外部准入控制器。
五.给客户端授权
创建minsvcbug目录存放配置文件。
root@k8scludes1:~# mkdir minsvcbug
root@k8scludes1:~# cd minsvcbug
root@k8scludes1:~/minsvcbug# pwd
/root/minsvcbug
创建minsvcbug命名空间。
root@k8scludes1:~/minsvcbug# kubectl create ns minsvcbug
namespace/minsvcbug created
切换命名空间。
root@k8scludes1:~/minsvcbug# kubens minsvcbug
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "minsvcbug".
root@k8scludes1:~/minsvcbug# kubectl get pod
No resources found in minsvcbug namespace.
本次使用etcd2机器作为k8s客户端，kctom为kubeconfig文件，里面的用户为tom,需要给tom授权，客户端才能具有相关访问权限。
关于kubeconfig文件的详细操作，请查看博客《Kubernetes(k8s)访问控制：身份认证》。
关于授权的详细操作，请查看博客《Kubernetes(k8s)访问控制：权限管理之RBAC鉴权》。
[root@etcd2 ~]# ls
kctom
查看minsvcbug命名空间下的pod，现在tom用户还没有minsvcbug命名空间的pod访问权限。
[root@etcd2 ~]# kubectl get pod --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): pods is forbidden: User "tom" cannot list resource "pods" in API group "" in the namespace "minsvcbug"
给客户端tom用户授予创建/删除/查看pod的权限，编辑角色role配置文件。
root@k8scludes1:~/minsvcbug# vim role1.yaml
#role1角色对pod,svc,secret具有get,list,watch,create,delete权限
root@k8scludes1:~/minsvcbug# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: role1
rules:
- apiGroups:
- ""
resources:
- pods
- services
- secrets
verbs:
- get
- list
- watch
- create
- delete
创建角色role1。
root@k8scludes1:~/minsvcbug# kubectl apply -f role1.yaml
role.rbac.authorization.k8s.io/role1 created
查看角色。
root@k8scludes1:~/minsvcbug# kubectl get role -o wide
NAME CREATED AT
role1 2022-05-20T03:42:48Z
查看role1的详细描述。
root@k8scludes1:~/minsvcbug# kubectl describe role role1
Name: role1
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods [] [] [get list watch create delete]
secrets [] [] [get list watch create delete]
services [] [] [get list watch create delete]
把角色role1绑定给tom用户。
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebindingtotom --role=role1 --user=tom
rolebinding.rbac.authorization.k8s.io/rolebindingtotom created
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
NAME ROLE AGE
rolebindingtotom Role/role1 11s
给tom用户授予创建/删除/查看pod的权限之后，在客户端查看。可以看到tom用户对minsvcbug命名空间的pod有访问权限了。
[root@etcd2 ~]# kubectl get pod --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
可以看到tom用户对default命名空间的pod没有访问权限。
[root@etcd2 ~]# kubectl get pod --kubeconfig=kctom
Error from server (Forbidden): pods is forbidden: User "tom" cannot list resource "pods" in API group "" in the namespace "default"
在客户端etcd2机器上创建目录存放文件。
[root@etcd2 ~]# mkdir minsvcbug
[root@etcd2 ~]# cd minsvcbug/
[root@etcd2 minsvcbug]# pwd
/root/minsvcbug
编辑pod配置文件，使用nginx创建pod。
[root@etcd2 minsvcbug]# cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
在客户端使用nginx镜像创建pod。
[root@etcd2 minsvcbug]# kubectl apply -f pod.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
pod创建成功，说明成功授权了。
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 36s 10.244.218.132 k8scludes2 <none> <none>
[root@etcd2 minsvcbug]# kubectl delete -f pod.yaml --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
现在tom用户就具备了minsvcbug命名空间的创建/删除/查看pod的权限了。
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
六.启用PodSecurityPolicy（PSP）
接下来启用PSP，编辑kube-apiserver.yaml文件。
root@k8scludes1:~/minsvcbug# ls /etc/kubernetes/manifests/kube-apiserver.yaml
/etc/kubernetes/manifests/kube-apiserver.yaml
启用PSP需要添加- --enable-admission-plugins=PodSecurityPolicy。
root@k8scludes1:~/minsvcbug# vim /etc/kubernetes/manifests/kube-apiserver.yaml
root@k8scludes1:~/minsvcbug# grep enable-admission-plugins /etc/kubernetes/manifests/kube-apiserver.yaml
- --enable-admission-plugins=NodeRestriction,PodSecurityPolicy
重启kubelet使配置生效。
root@k8scludes1:~/minsvcbug# systemctl restart kubelet
root@k8scludes1:~/minsvcbug# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8scludes1 Ready control-plane,master 33d v1.22.2
k8scludes2 Ready <none> 33d v1.22.2
k8scludes3 Ready <none> 33d v1.22.2
PSP(pod security policy)默认没有开启，如果开启PSP之后，没有写PSP规则的话，任何用户都创建不了POD，包括管理员。
注意：一定要等k8s正常工作之后再启动PSP，如果etcd,api都没启动好就启动PSP，则相关pod都启动不了了 ，等PSP启动之后再去写PSP规则。
启用PSP之后，创建pod必须先访问PSP规则，通过了PSP规则才能创建POD，否则创建pod失败。
默认情况下，普通用户没有访问PSP规则的权限，管理员用户可以直接访问PSP规则，创建好PSP规则之后，对普通用户要做一个RBAC的权限设置，使其可以访问某个PSP规则，再决定是不是能够把pod创建出 来(1.是否有足够的RBAC权限 2.创建的POD是否符合PSP规则)。
七.PSP规则之禁止创建特权用户pod
查看PSP规则。
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
如果没有任何PSP规则，则任何用户都不能创建pod。
编辑PSP规则，我们的目的是拒绝特权pod运行，privileged: false 表示不允许特权pod运行。
root@k8scludes1:~/minsvcbug# vim psp-deny-privileges-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-deny-privileges-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-deny-privileges-pod
spec:
# Don't allow privileged pods!
privileged: false
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- '*'
创建psp。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-deny-privileges-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-deny-privileges-pod created
psp-deny-privileges-pod规则就创建好了。
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-deny-privileges-pod false RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
创建psp-deny-privileges-pod的PSP规则之后,在客户端使用普通用户创建pod。
编辑pod配置文件。
[root@etcd2 minsvcbug]# cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
创建pod失败，就算普通用户tom创建没有特权用户的pod，因为tom没有访问PSP规则的权限，所以创建pod失败。
[root@etcd2 minsvcbug]# kubectl apply -f pod.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "pod.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: []
[root@etcd2 minsvcbug]# kubectl get pod --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
下面要给tom用户授予PSP的访问权限，对PSP的授权可以是clusterrole，也可以是role，PSP规则是全局生效的，在其他命名空间也可以看到，本次使用clusterrole授权。
生成clusterrole-access-psp-rule的yaml文件，表示：角色clusterrole-access-psp-rule对名为psp-deny-privileges-pod有use权限。
root@k8scludes1:~# kubectl create clusterrole clusterrole-access-psp-rule --verb=use --resource=psp --resource-name=psp-deny-privileges-pod --dry-run=client -o yaml >clusterrole-access-psp-rule.yaml
生成的yaml文件如下：
root@k8scludes1:~# vim clusterrole-access-psp-rule.yaml
root@k8scludes1:~# cat clusterrole-access-psp-rule.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: clusterrole-access-psp-rule
rules:
- apiGroups:
- policy
resourceNames:
- psp-deny-privileges-pod
resources:
- podsecuritypolicies
verbs:
- use
创建clusterrole-access-psp-rule。
root@k8scludes1:~# kubectl apply -f clusterrole-access-psp-rule.yaml
clusterrole.rbac.authorization.k8s.io/clusterrole-access-psp-rule created
root@k8scludes1:~# kubectl get clusterrole | grep clusterrole-access-psp-rule
clusterrole-access-psp-rule 2022-05-20T10:21:15Z
查看角色的详细描述。
root@k8scludes1:~# kubectl describe clusterrole clusterrole-access-psp-rule
Name: clusterrole-access-psp-rule
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
podsecuritypolicies.policy [] [psp-deny-privileges-pod] [use]
把角色clusterrole-access-psp-rule绑定给tom用户。
root@k8scludes1:~# kubectl create rolebinding rolebinding-clusterrole-access-psp --clusterrole=clusterrole-access-psp-rule --user=tom
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-access-psp created
root@k8scludes1:~# kubectl get rolebinding -o wide
NAME ROLE AGE USERS GROUPS SERVICEACCOUNTS
rolebinding-clusterrole-access-psp ClusterRole/clusterrole-access-psp-rule 15s tom
rolebindingtotom Role/role1 6h40m tom
给tom用户授予PSP的访问权限之后，到客户端创建无特权用户的pod。
[root@etcd2 minsvcbug]# kubectl apply -f pod.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
pod创建成功，现在tom用户就可以创建无特权用户的pod了。
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 21s 10.244.218.145 k8scludes2 <none> <none>
删除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
编辑pod文件，securityContext:privileged: true 表示创建特权用户的pod。
[root@etcd2 minsvcbug]# cat podprivileged.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
#创建特权pod
securityContext:
privileged: true
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
创建pod失败，Privileged containers are not allowed 创建特权容器是不允许的。
[root@etcd2 minsvcbug]# kubectl apply -f podprivileged.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podprivileged.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
删除psp-deny-privileges-pod规则。
root@k8scludes1:~/minsvcbug# kubectl delete psp psp-deny-privileges-pod
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy "psp-deny-privileges-pod" deleted
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
删除clusterrole-access-psp-rule(不删除也可以)。
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep clusterrole-access-psp-rule
clusterrole-access-psp-rule 2022-05-20T10:21:15Z
root@k8scludes1:~/minsvcbug# kubectl delete clusterrole clusterrole-access-psp-rule
clusterrole.rbac.authorization.k8s.io "clusterrole-access-psp-rule" deleted
删除rolebinding-clusterrole-access-psp(不删除也可以)。
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebinding-clusterrole-access-psp
rolebinding.rbac.authorization.k8s.io "rolebinding-clusterrole-access-psp" deleted
总结：
- 在启用了PSP之后，没有任何PSP规则的情况下，任何用户(包括管理员)都不能创建pod；
- 在启用了PSP之后，且创建了PSP规则，要确保用户能访问到此PSP规则，创建clusterrole，让用户能use某条PSP规则；
- 能不能成功创建pod，决定权在于是否具备创建pod的RBAC权限，有对应的RBAC权限之后，创建的pod是否满足PSP规则。
八.PSP规则之数据卷
8.1 限定数据卷使用某一个目录
编辑psp配置文件，privileged: true表示可以创建特权用户的pod，volumes:- '*' 表示可以使用所有类型的数据卷，allowedHostPaths:- pathPrefix: "/tmp" 表示数据卷只能挂载/tmp目录，psp-mount-pod这条PSP规则规定hostPath只能使用/tmp目录。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- '*'
allowedHostPaths:
- pathPrefix: "/tmp"
创建PSP。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod created
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
创建了新的PSP规则之后，需要重新给客户端的tom用户授权。
创建clusterrole--psp-mount-rule，角色clusterrole--psp-mount-rule对名为psp-mount-pod的psp规则有use权限。
root@k8scludes1:~/minsvcbug# kubectl create clusterrole clusterrole--psp-mount-rule --verb=use --resource=psp --resource-name=psp-mount-pod
clusterrole.rbac.authorization.k8s.io/clusterrole--psp-mount-rule created
把角色clusterrole--psp-mount-rule绑定给tom用户。
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebinding-clusterrole-mount-psp --clusterrole=clusterrole--psp-mount-rule --user=tom
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-mount-psp created
给客户端tom用户授权之后，编辑pod配置文件，volumes:- name: v1 hostPath: path: /opt 表示创建名叫v1的数据卷，目录为/opt ,把v1这个数据卷挂载到容器的/data目录。securityContext:privileged: true表示创建特权pod。
关于数据卷的详细操作，请查看博客《Kubernetes(k8s)存储管理之数据卷volumes(一)：volumes的引入和emptyDir数据卷》。
[root@etcd2 minsvcbug]# vim podmount.yaml
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
#数据卷v1
volumes:
- name: v1
hostPath:
path: /opt
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
#把数据卷v1挂载到/data目录
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
创建pod失败，"/opt": is not allowed to be used] ，/opt目录不被允许使用。
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podmount.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.volumes[0].hostPath.pathPrefix: Invalid value: "/opt": is not allowed to be used]
修改pod配置文件，把v1数据卷的/opt目录换成/tmp目录。
[root@etcd2 minsvcbug]# vim podmount.yaml
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
volumes:
- name: v1
hostPath:
path: /tmp
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
创建pod。
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
把v1数据卷的/opt目录换成/tmp目录之后，pod就创建成功了。
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 14s 10.244.218.134 k8scludes2 <none> <none>
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
8.2 限定数据卷的类型为emptyDir
编辑PSP配置文件，volumes:- 'emptyDir' 限定数据卷的类型为emptyDir。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
#allowedHostPaths:
#- pathPrefix: "/tmp"
创建PSP。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
编辑pod配置文件，hostPath:path: /tmp 指定hostPath目录，这里使用的是hostPath类型的数据卷。
root@k8scludes1:~/minsvcbug# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
hostPath:
path: /tmp
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
创建pod失败，Invalid value: "hostPath": hostPath volumes are not allowed to be used 表示不能使用hostPath数据卷。
root@k8scludes1:~/minsvcbug# kubectl apply -f podmount.yaml
Error from server (Forbidden): error when creating "podmount.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.volumes[1]: Invalid value: "projected": projected volumes are not allowed to be used spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used]
这次在客户端创建pod。
修改pod配置文件，emptyDir: {}表示使用emptyDir数据卷。
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
创建pod失败，pod的数据卷类型修改为emptyDir之后，还是报错：Invalid value: "projected": projected volumes are not allowed to be used]。
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podmount.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.volumes[1]: Invalid value: "projected": projected volumes are not allowed to be used]
先修改PSP规则，数据卷类型还是设置为所有类型volumes:- "*" 。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- "*"
#- 'emptyDir'
#allowedHostPaths:
#- pathPrefix: "/tmp"
应用PSP。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
PSP规则的数据卷类型修改为所有类型之后，创建pod成功。
root@k8scludes1:~/minsvcbug# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
root@k8scludes1:~/minsvcbug# kubectl apply -f podmount.yaml
pod/podtest created
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 9s 10.244.218.155 k8scludes2 <none> <none>
使用yaml文件的格式查看podtest，可以发现有两个数据卷，一个是emptyDir类型的，一个是projected类型的。
root@k8scludes1:~/minsvcbug# kubectl get pod podtest -o yaml
apiVersion: v1
kind: Pod
metadata:
......
volumes:
- emptyDir: {}
name: v1
- name: kube-api-access-8xjp8
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
......
更新软件源。
root@k8scludes1:~/minsvcbug# apt-get update
安装jq,jq可以使JSON格式更好看。
root@k8scludes1:~/minsvcbug# apt-get install jq -y
root@k8scludes1:~/minsvcbug# which jq
/usr/bin/jq
以json格式的方式查看podtest，jsonpath='{.spec.volumes}'表示只查看spec字段下的volumes，这样显示不直观，使用jq使JSON格式化。
root@k8scludes1:~/minsvcbug# kubectl get pod podtest -o jsonpath='{.spec.volumes}'
[{"emptyDir":{},"name":"v1"},{"name":"kube-api-access-8xjp8","projected":{"defaultMode":420,"sources":[{"serviceAccountToken":{"expirationSeconds":3607,"path":"token"}},{"configMap":{"items":[{"key":"ca.crt","path":"ca.crt"}],"name":"kube-root-ca.crt"}},{"downwardAPI":{"items":[{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"},"path":"namespace"}]}}]}}]root@k8scludes1:~/minsvcbug#
可以发现不仅有emptyDir数据卷，还有projected类型的数据卷。
root@k8scludes1:~/minsvcbug# kubectl get pod podtest -o jsonpath='{.spec.volumes}' | jq
[
{
"emptyDir": {},
"name": "v1"
},
{
"name": "kube-api-access-8xjp8",
"projected": {
"defaultMode": 420,
"sources": [
{
"serviceAccountToken": {
"expirationSeconds": 3607,
"path": "token"
}
},
{
"configMap": {
"items": [
{
"key": "ca.crt",
"path": "ca.crt"
}
],
"name": "kube-root-ca.crt"
}
},
{
"downwardAPI": {
"items": [
{
"fieldRef": {
"apiVersion": "v1",
"fieldPath": "metadata.namespace"
},
"path": "namespace"
}
]
}
}
]
}
}
]
所以创建PSP规则的时候不仅要指定emptyDir类型的数据卷，还要指定projected类型的数据卷，修改PSP规则。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
应用PSP规则。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
删除pod。
root@k8scludes1:~/minsvcbug# kubectl get pod
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 13m
root@k8scludes1:~/minsvcbug# kubectl delete pod podtest
pod "podtest" deleted
修改pod配置文件，数据卷类型为emptyDir。
[root@etcd2 minsvcbug]# cat podmount.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
客户端继续创建pod，在客户端成功创建emptyDir类型数据卷的pod。
[root@etcd2 minsvcbug]# kubectl apply -f podmount.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 6s 10.244.218.136 k8scludes2 <none> <none>
删除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
No resources found in minsvcbug namespace.
九.PSP规则之指定使用宿主机网络
编辑pod配置文件，hostNetwork: true 表示是否使用宿主机网络，true表示使用宿主机网络，默认是false，不使用宿主机网络。
hostNetwork: true相当于docker run -dit --name=nginx --network=net nginx 共享宿主机网络空间。
[root@etcd2 minsvcbug]# vim podnetwork.yaml
[root@etcd2 minsvcbug]# cat podnetwork.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
#hostNetwork: true 表示pod使用宿主机网络
hostNetwork: true
restartPolicy: Always
status: {}
创建pod失败，Invalid value: true: Host network is not allowed to be used]表示 pod不能使用宿主机网络。
[root@etcd2 minsvcbug]# kubectl apply -f podnetwork.yaml --kubeconfig=kctom -n minsvcbug
Error from server (Forbidden): error when creating "podnetwork.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used]
修改PSP规则，使pod可以使用宿主机网络，hostNetwork: true 表示可以使用宿主机网络。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
应用PSP规则。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
客户端再次创建使用宿主机网络的pod，这次pod创建成功。
[root@etcd2 minsvcbug]# kubectl apply -f podnetwork.yaml --kubeconfig=kctom -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide --kubeconfig=kctom -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 17s 192.168.110.129 k8scludes2 <none> <none>
删除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest --kubeconfig=kctom -n minsvcbug
pod "podtest" deleted
在default命名空间，创建宿主机网络的pod。
root@k8scludes1:~/minsvcbug# kubectl apply -f podnetwork.yaml
pod/podtest created
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 7s 192.168.110.129 k8scludes2 <none> <none>
进入pod。
#可以看到pod里使用的是宿主机里的网络
root@k8scludes1:~/minsvcbug# kubectl exec -it podtest -- bash
root@k8scludes2:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:71:45:c1 brd ff:ff:ff:ff:ff:ff
inet 192.168.110.129/24 brd 192.168.110.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe71:45c1/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:fd:bc:7b:4b brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
6: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
inet 10.244.218.128/32 scope global tunl0
valid_lft forever preferred_lft forever
#退出pod
root@k8scludes2:/# exit
exit
删除pod。
root@k8scludes1:~/minsvcbug# kubectl delete pod podtest
pod "podtest" deleted
root@k8scludes1:~/minsvcbug# kubectl get pod
No resources found in minsvcbug namespace.
十.PSP规则之pod使用特定的UID运行
编辑pod配置文件，hostNetwork: true 表示pod使用宿主机网络 。
root@k8scludes1:~/minsvcbug# cat podnetwork.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
hostNetwork: true
restartPolicy: Always
status: {}
创建pod。
root@k8scludes1:~/minsvcbug# kubectl apply -f podnetwork.yaml
pod/podtest created
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 12s 192.168.110.129 k8scludes2 <none> <none>
进入pod，可以发现pod默认是以root身份运行的，可不可以设置pod以其他身份运行呢？
root@k8scludes1:~/minsvcbug# kubectl exec -it podtest -- bash
#查看网络，pod使用的是宿主机的网络
root@k8scludes2:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens32: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:71:45:c1 brd ff:ff:ff:ff:ff:ff
inet 192.168.110.129/24 brd 192.168.110.255 scope global ens32
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe71:45c1/64 scope link
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:4c:05:bc:65 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
6: tunl0@NONE: <NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
inet 10.244.218.128/32 scope global tunl0
valid_lft forever preferred_lft forever
#pod是以root用户运行的
root@k8scludes2:/# id
uid=0(root) gid=0(root) groups=0(root)
#退出pod
root@k8scludes2:/# exit
exit
查看PSP规则。
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
编辑PSP规则，runAsUser:rule: RunAsAny 中 runAsUser表示以某种身份运行，RunAsAny表示可以以任何身份运行，rule: MustRunAs ranges:- min: 1000 max: 2000 表示pod必须以1000-2000的UID身份运行。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
#rule: RunAsAny
rule: MustRunAs
#MustRunAs 表示pod必须以1000-2000的UID身份运行
ranges:
- min: 1000
max: 2000
fsGroup:
rule: RunAsAny
volumes:
- 'emptyDir'
- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
使PSP规则生效。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny MustRunAs RunAsAny RunAsAny false emptyDir,projected
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
因为客户端etcd2需要进入到pod里，以及查看pod的日志，再给客户端添加pods/exec， pods/log权限，直接在Role配置文件中加入即可。
root@k8scludes1:~/minsvcbug# vim role1.yaml
root@k8scludes1:~/minsvcbug# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: role1
rules:
- apiGroups:
- ""
resources:
- pods
#pods/exec， pods/log是新加的权限
- pods/exec
- pods/log
- services
- secrets
verbs:
- get
- list
- watch
- create
- delete
应用Role。
root@k8scludes1:~/minsvcbug# kubectl apply -f role1.yaml
role.rbac.authorization.k8s.io/role1 configured
查看role1的权限信息。
root@k8scludes1:~/minsvcbug# kubectl describe role role1
Name: role1
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods/exec [] [] [get list watch create delete]
pods/log [] [] [get list watch create delete]
pods [] [] [get list watch create delete]
secrets [] [] [get list watch create delete]
services [] [] [get list watch create delete]
接下来在客户端etcd2机器进行测试。
把kubeconfig文件kctom复制到~/.kube/config，这样执行命令行就不用每次都指定--kubeconfig了。
[root@etcd2 minsvcbug]# cp kctom ~/.kube/config
[root@etcd2 minsvcbug]# ls ~/.kube/
cache config
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
No resources found in minsvcbug namespace.
编辑pod文件。
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
#emptyDir类型的数据卷
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: podtest
#创建特权pod
securityContext:
privileged: true
resources: {}
#数据卷挂载
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
#hostNetwork: true 表示pod使用宿主机网络
hostNetwork: true
restartPolicy: Always
status: {}
创建pod。
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
pod/podtest created
pod创建失败。
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
NAME READY STATUS RESTARTS AGE
podtest 0/1 CrashLoopBackOff 1 (5s ago) 6s
查看pod日志：podtest里不允许root来运行，podtest默认运行的是nginx进程，nginx进程需要使用root身份来运行。
[root@etcd2 minsvcbug]# kubectl logs podtest -n minsvcbug
2022/05/24 07:53:04 [warn] 1#1: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
2022/05/24 07:53:04 [emerg] 1#1: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
删除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest -n minsvcbug
pod "podtest" deleted
编辑pod配置文件，这次pod不运行nginx的默认进程，因为nginx进程需要root才能正常运行，我们运行sleep 100000命令，sleep 100000命令什么用户都可以运行。
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
#运行sleep 100000命令
command: ["sh","-c","sleep 100000"]
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
hostNetwork: true
restartPolicy: Always
status: {}
创建pod。
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 12s 192.168.110.129 k8scludes2 <none> <none>
进入pod。
[root@etcd2 minsvcbug]# kubectl exec -it podtest -n minsvcbug -- bash
#现在默认的用户UID为1000
I have no name!@k8scludes2:/$ id
uid=1000 gid=0(root) groups=0(root)
#退出pod
I have no name!@k8scludes2:/$ exit
exit
删除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
No resources found in minsvcbug namespace.
编辑pod配置文件，securityContext:runAsUser: 3000 表示pod以UID为3000的身份运行。
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
securityContext:
runAsUser: 3000
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
command: ["sh","-c","sleep 100000"]
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
创建pod失败，UID 3000不在PSP规则规定的{1000 2000}之间，所以创建pod失败。
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
Error from server (Forbidden): error when creating "poduid.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.runAsUser: Invalid value: 3000: must be in the ranges: [{1000 2000}]]
编辑pod配置文件，securityContext:runAsUser: 1500 表示pod以UID为1500的身份运行。
[root@etcd2 minsvcbug]# vim poduid.yaml
[root@etcd2 minsvcbug]# cat poduid.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
securityContext:
runAsUser: 1500
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
command: ["sh","-c","sleep 100000"]
name: podtest
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
创建pod成功。
[root@etcd2 minsvcbug]# kubectl apply -f poduid.yaml -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 12s
进入pod。
[root@etcd2 minsvcbug]# kubectl exec -it podtest -n minsvcbug -- bash
#UID 1500在{1000 2000}范围，所以pod运行成功，pod运行进程UID也为1500
I have no name!@podtest:/$ id
uid=1500 gid=0(root) groups=0(root)
#退出pod
I have no name!@podtest:/$ exit
exit
十一.PSP规则之指定hostport端口
修改一下vim设置，设置为粘贴模式之后，粘贴格式看着更规范。
root@k8scludes1:~/minsvcbug# vim /etc/vim/vimrc
root@k8scludes1:~/minsvcbug# tail -1 /etc/vim/vimrc
set paste
编辑PSP规则，因为nginx进程需要root身份才能运行，RUNASUSER 变为RunAsAny，RunAsAny表示可以以任何身份运行。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
应用PSP规则。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
去客户端创建pod。
编辑pod配置文件，containerPort: 80 表示容器端口80，hostPort: 8080表示宿主机端口8080 ，表示把nginx容器的80端口映射为宿主机的8080端口。
这样创建的pod类似于docker端口映射：docker run -p 80 nginx 只写一个端口80的话表示容器端口，docker run -p 80:80 nginx 80:80 表示宿主机端口:容器端口。
[root@etcd2 minsvcbug]# vim podport.yaml
[root@etcd2 minsvcbug]# cat podport.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
volumeMounts:
- name: v1
mountPath: /data
ports:
- name: http
containerPort: 80
hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
创建pod的时候发现tom用户没有pods的patch权限，我们去授权一下。
[root@etcd2 minsvcbug]# kubectl apply -f podport.yaml -n minsvcbug
for: "podport.yaml": pods "podtest" is forbidden: User "tom" cannot patch resource "pods" in API group "" in the namespace "minsvcbug"
修改Role配置文件，添加patch权限。
root@k8scludes1:~/minsvcbug# vim role1.yaml
root@k8scludes1:~/minsvcbug# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: role1
rules:
- apiGroups:
- ""
resources:
- pods
- pods/exec
- pods/log
- services
- secrets
verbs:
- get
- list
- watch
- create
- delete
#添加patch权限
- patch
应用角色。
root@k8scludes1:~/minsvcbug# kubectl apply -f role1.yaml
role.rbac.authorization.k8s.io/role1 configured
客户端继续创建pod。
创建pod失败，Invalid value: 8080: Host port 8080 is not allowed to be used. Allowed ports: [] 表示PSP规则规定的Host port为空，不能使用hostport端口，我们需要修改PSP规则。
[root@etcd2 minsvcbug]# kubectl get pod -n minsvcbug
No resources found in minsvcbug namespace.
[root@etcd2 minsvcbug]# kubectl apply -f podport.yaml -n minsvcbug
Error from server (Forbidden): error when creating "podport.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].hostPort: Invalid value: 8080: Host port 8080 is not allowed to be used. Allowed ports: []]
修改PSP规则，hostPorts:- min: 1000 max: 8090表示hostport的端口范围为1000-8090。
root@k8scludes1:~/minsvcbug# vim psp-mount-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-mount-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-mount-pod
spec:
#allow privileged pods!
privileged: true
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
#指定hostport的端口范围为1000-8090
hostPorts:
- min: 1000
max: 8090
应用PSP规则。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-mount-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-mount-pod configured
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
现在客户端创建pod就成功了。
[root@etcd2 minsvcbug]# kubectl apply -f podport.yaml -n minsvcbug
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n minsvcbug
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 53s 10.244.218.150 k8scludes2 <none> <none>
访问nginx的方法为：curl k8scludes2的ip:8080，nginx访问成功。
[root@etcd2 minsvcbug]# curl 192.168.110.129:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
删除pod。
[root@etcd2 minsvcbug]# kubectl delete pod podtest -n minsvcbug
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n minsvcbug
No resources found in minsvcbug namespace.
删除rolebinding。
root@k8scludes1:~# cd minsvcbug/
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
NAME ROLE AGE
rolebinding-clusterrole-mount-psp ClusterRole/clusterrole--psp-mount-rule 3d21h
rolebindingtotom Role/role1 5d3h
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebinding-clusterrole-mount-psp
rolebinding.rbac.authorization.k8s.io "rolebinding-clusterrole-mount-psp" deleted
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
NAME ROLE AGE
rolebindingtotom Role/role1 5d3h
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebindingtotom
rolebinding.rbac.authorization.k8s.io "rolebindingtotom" deleted
root@k8scludes1:~/minsvcbug# kubectl get rolebinding
No resources found in minsvcbug namespace.
删除role。
root@k8scludes1:~/minsvcbug# kubectl get role
NAME CREATED AT
role1 2022-05-20T03:42:48Z
root@k8scludes1:~/minsvcbug# kubectl delete role role1
role.rbac.authorization.k8s.io "role1" deleted
root@k8scludes1:~/minsvcbug# kubectl get role
No resources found in minsvcbug namespace.
删除PSP规则。
root@k8scludes1:~/minsvcbug# kubectl get psp
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-mount-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
#删除psp-mount-pod规则
root@k8scludes1:~/minsvcbug# kubectl delete psp psp-mount-pod
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy "psp-mount-pod" deleted
十二.PSP规则实战1
我们现在要解决的问题是：有三个命名空间ns1,ns2,ns3，tom用户只有在ns1命名空间里可以创建特权用户pod，其他命名空间都不能创建特权用户pod，需要怎么做？
创建3个命名空间。
root@k8scludes1:~/minsvcbug# kubectl create ns ns1
namespace/ns1 created
root@k8scludes1:~/minsvcbug# kubectl create ns ns2
namespace/ns2 created
root@k8scludes1:~/minsvcbug# kubectl create ns ns3
namespace/ns3 created
创建一个普通的RBAC权限，让客户端tom用户可以在所有的命名空间里创建和删除pod,使用clusterrole和clusterrolebinding。
编辑ClusterRole配置文件，clusterole1具有对pod的查询/创建/删除权限，因为需要看日志和进入到pod里，添加了pods/exec和pods/log资源。
root@k8scludes1:~/minsvcbug# vim clusterrole1.yaml
root@k8scludes1:~/minsvcbug# cat clusterrole1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: clusterole1
rules:
- apiGroups:
- ""
resources:
- pods
- pods/exec
- pods/log
verbs:
- get
- list
- watch
- create
- delete
- patch
应用ClusterRole。
root@k8scludes1:~/minsvcbug# kubectl apply -f clusterrole1.yaml
clusterrole.rbac.authorization.k8s.io/clusterole1 created
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep clusterole1
clusterole1 2022-05-25T07:31:28Z
查看clusterole1权限信息。
root@k8scludes1:~/minsvcbug# kubectl describe clusterrole clusterole1
Name: clusterole1
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods/exec [] [] [get list watch create delete patch]
pods/log [] [] [get list watch create delete patch]
pods [] [] [get list watch create delete patch]
把clusterole1绑定给tom用户，clusterole1和clusterrolebinding1的目的是让tom用户可以在所有命名空间里创建和删除pod。
现在客户端的tom用户在所有命名空间都可以创建和删除pod了。
root@k8scludes1:~/minsvcbug# kubectl create clusterrolebinding clusterrolebinding1 --clusterrole=clusterole1 --user=tom
clusterrolebinding.rbac.authorization.k8s.io/clusterrolebinding1 created
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding | grep clusterrolebinding1
clusterrolebinding1 ClusterRole/clusterole1 21s
创建两条psp规则，一条PSP规则禁止创建特权用户pod，另外一条PSP规则允许创建特权用户pod。
编辑psp-allowpri-pod规则，psp-allowpri-pod规则允许创建特权用户pod，privileged: true。
root@k8scludes1:~/minsvcbug# vim psp-allowpri-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-allowpri-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-allowpri-pod
spec:
#allow privileged pods!
privileged: true
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
#hostPorts:
#- min: 1000
# max: 8090
编辑psp-denypri-pod规则，psp-allowpri-pod规则禁止创建特权用户pod，privileged: false。
root@k8scludes1:~/minsvcbug# vim psp-denypri-pod.yaml
root@k8scludes1:~/minsvcbug# cat psp-denypri-pod.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp-denypri-pod
spec:
#allow privileged pods!
privileged: false
#hostNetwork: true
# The rest fills in some required fields.
seLinux:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
runAsUser:
rule: RunAsAny
#rule: MustRunAs
#ranges:
#- min: 1000
# max: 2000
fsGroup:
rule: RunAsAny
volumes:
- '*'
#- 'emptyDir'
#- 'projected'
#allowedHostPaths:
#- pathPrefix: "/tmp"
#hostPorts:
#- min: 1000
# max: 8090
创建PSP规则。
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-allowpri-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-allowpri-pod created
root@k8scludes1:~/minsvcbug# kubectl apply -f psp-denypri-pod.yaml
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
podsecuritypolicy.policy/psp-denypri-pod created
root@k8scludes1:~/minsvcbug# kubectl get psp -o wide
Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP READONLYROOTFS VOLUMES
controller false RunAsAny MustRunAs MustRunAs MustRunAs true configMap,secret,emptyDir
psp-allowpri-pod true RunAsAny RunAsAny RunAsAny RunAsAny false *
psp-denypri-pod false RunAsAny RunAsAny RunAsAny RunAsAny false *
speaker true NET_RAW RunAsAny RunAsAny RunAsAny RunAsAny true configMap,secret,emptyDir
为了达到tom用户在所有的namespace里都不能创建特权用户pod的效果：需要通过clusterrole和clusterrolebinding把psp-denypri-pod规则授权给tom用户。
现在的clusterrole，clusterrolebinding如下：
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep cluste
cert-manager-controller-clusterissuers 2022-04-24T17:07:47Z
cluster-admin 2022-04-16T14:57:55Z
clusterole1 2022-05-25T07:31:28Z
clusterrole--psp-mount-rule 2022-05-21T09:42:10Z
system:controller:clusterrole-aggregation-controller 2022-04-16T14:57:55Z
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding | grep cluster
cert-manager-controller-clusterissuers ClusterRole/cert-manager-controller-clusterissuers 30d
cluster-admin ClusterRole/cluster-admin 38d
clusterrolebinding1 ClusterRole/clusterole1 30m
system:controller:clusterrole-aggregation-controller ClusterRole/system:controller:clusterrole-aggregation-controller 38d
创建clusterrole，角色clusterrole--psp-denypri-rule对名为psp-denypri-pod的psp规则有use权限。
root@k8scludes1:~/minsvcbug# kubectl create clusterrole clusterrole--psp-denypri-rule --verb=use --resource=psp --resource-name=psp-denypri-pod
clusterrole.rbac.authorization.k8s.io/clusterrole--psp-denypri-rule created
查看clusterrole--psp-denypri-rule的描述信息。
root@k8scludes1:~/minsvcbug# kubectl describe clusterrole clusterrole--psp-denypri-rule
Name: clusterrole--psp-denypri-rule
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
podsecuritypolicies.policy [] [psp-denypri-pod] [use]
把角色clusterrole--psp-denypri-rule绑定给tom用户，clusterrole--psp-denypri-rule和clusterrolebinding-clusterrole-denypri-psp的目的是让tom具有psp-denypri-pod规则的use权限，这样tom在所有的命名空间就不能创建特权用户pod了。
root@k8scludes1:~/minsvcbug# kubectl create clusterrolebinding clusterrolebinding-clusterrole-denypri-psp --clusterrole=clusterrole--psp-denypri-rule --user=tom
clusterrolebinding.rbac.authorization.k8s.io/clusterrolebinding-clusterrole-denypri-psp created
在客户端进行验证，首先测试在多个namespace创建和删除pod。
编辑pod配置文件，这是非特权用户pod。
[root@etcd2 minsvcbug]# vim podnotpri.yaml
[root@etcd2 minsvcbug]# cat podnotpri.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
resources: {}
volumeMounts:
- name: v1
mountPath: /data
#ports:
#- name: http
# containerPort: 80
# hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
tom用户在ns1，ns2，ns3都能创建非特权pod。
[root@etcd2 minsvcbug]# kubectl apply -f podnotpri.yaml -n ns1
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -n ns1
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 12s
[root@etcd2 minsvcbug]# kubectl apply -f podnotpri.yaml -n ns2
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -n ns2
NAME READY STATUS RESTARTS AGE
podtest 1/1 Running 0 7s
[root@etcd2 minsvcbug]# kubectl apply -f podnotpri.yaml -n ns3
pod/podtest created
tom用户在ns1，ns2，ns3都能删除pod。
[root@etcd2 minsvcbug]# kubectl delete -f podnotpri.yaml -n ns1
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl delete -f podnotpri.yaml -n ns2
pod "podtest" deleted
#这样tom用户就可以在ns1，ns2，ns3创建和删除pod了
[root@etcd2 minsvcbug]# kubectl delete -f podnotpri.yaml -n ns3
pod "podtest" deleted
编辑pod配置文件，securityContext:privileged: true 表示创建特权用户pod。
[root@etcd2 minsvcbug]# vim podpri.yaml
[root@etcd2 minsvcbug]# cat podpri.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
#securityContext:privileged: true 表示创建特权用户pod
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
#ports:
#- name: http
# containerPort: 80
# hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
可以发现，tom用户可以在ns1，ns2，ns3命名空间创建pod，但是不能创建特权pod。
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns1
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns2
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns3
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
创建clusterrole，角色clusterrole--psp-allowpri-rule对名为psp-allowpri-pod的psp规则有use权限。
root@k8scludes1:~/minsvcbug# kubectl create clusterrole clusterrole--psp-allowpri-rule --verb=use --resource=psp --resource-name=psp-allowpri-pod
clusterrole.rbac.authorization.k8s.io/clusterrole--psp-allowpri-rule created
把角色clusterrole--psp-allowpri-rule绑定给tom用户，这样tom用户既可以访问psp-allowpri-pod规则，也可以访问psp-denypri-pod规则。
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebinding-clusterrole-allowpri-psp --clusterrole=clusterrole--psp-allowpri-rule --user=tom
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-allowpri-psp created
删除rolebinding-clusterrole-allowpri-psp，因为没有指定是哪个命名空间，不指定的话默认是当前命名空间。
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebinding-clusterrole-allowpri-psp
rolebinding.rbac.authorization.k8s.io "rolebinding-clusterrole-allowpri-psp" deleted
重新进行角色绑定，这样tom用户在ns1命名空间既可以访问psp-allowpri-pod规则，也可以访问psp-denypri-pod规则。
注意：对于PSP规则，拒绝和允许同时出现的话，允许优先级比较高。
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebinding-clusterrole-allowpri-psp --clusterrole=clusterrole--psp-allowpri-rule --user=tom -n ns1
rolebinding.rbac.authorization.k8s.io/rolebinding-clusterrole-allowpri-psp created
下面在客户端进行测试。
编辑pod配置文件，下面的配置文件是为了创建特权用户pod。
[root@etcd2 minsvcbug]# cat podpri.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: podtest
name: podtest
spec:
terminationGracePeriodSeconds: 0
volumes:
- name: v1
emptyDir: {}
containers:
- image: hub.c.163.com/library/nginx:latest
imagePullPolicy: IfNotPresent
name: podtest
#创建特权pod
securityContext:
privileged: true
resources: {}
volumeMounts:
- name: v1
mountPath: /data
#ports:
#- name: http
# containerPort: 80
# hostPort: 8080
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
在ns1命名空间创建特权pod成功。
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns1
pod/podtest created
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns1
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
podtest 1/1 Running 0 12s 10.244.218.148 k8scludes2 <none> <none>
在ns2，ns3命名空间创建特权pod失败。
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns2
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@etcd2 minsvcbug]# kubectl apply -f podpri.yaml -n ns3
Error from server (Forbidden): error when creating "podpri.yaml": pods "podtest" is forbidden: PodSecurityPolicy: unable to admit pod: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
这样就实现了tom用户可以在ns1命名空间创建特权用户pod，在其他命名空间都不可以创建特权pod的需求。
清理一下pod。
[root@etcd2 minsvcbug]# kubectl delete -f podpri.yaml -n ns1
pod "podtest" deleted
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns1
No resources found in ns1 namespace.
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns2
No resources found in ns2 namespace.
[root@etcd2 minsvcbug]# kubectl get pod -o wide -n ns3
No resources found in ns3 namespace.
十三.PSP规则实战2:创建deployment
现在没有deploy，使用hub.c.163.com/library/nginx:latest镜像创建一个deploy，先生成yaml文件。关于deploy的详细操作，请查看博客《Kubernetes(k8s)控制器(一)：deployment》。
root@k8scludes1:~/minsvcbug# kubectl get deploy
No resources found in minsvcbug namespace.
root@k8scludes1:~/minsvcbug# kubectl create deployment nginxdeploy --image=hub.c.163.com/library/nginx:latest --dry-run=client -o yaml >nginxdeploy.yaml
生成的deploy配置文件如下：
root@k8scludes1:~/minsvcbug# vim nginxdeploy.yaml
root@k8scludes1:~/minsvcbug# cat nginxdeploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
#这个labels是deployment的标签
labels:
app: nginxdeploy
#deployment名字
name: nginxdeploy
spec:
#replicas: 1 表示副本数为1，只生成一个pod
replicas: 1
#selector标签选择器
selector:
matchLabels:
app: nginxdeploy
strategy: {}
#template下面是pod的模板
template:
metadata:
creationTimestamp: null
#这个labels表示pod标签
labels:
app: nginxdeploy
spec:
#当需要关闭容器时，立即杀死容器而不等待默认的30秒优雅停机时长。
terminationGracePeriodSeconds: 0
containers:
- image: hub.c.163.com/library/nginx:latest
#imagePullPolicy: IfNotPresent：表示如果本地已经存在该镜像，则不重新下载；否则从远程 Docker Hub 下载该镜像
imagePullPolicy: IfNotPresent
name: nginx
resources: {}
status: {}
创建deploy。
root@k8scludes1:~/minsvcbug# kubectl apply -f nginxdeploy.yaml
deployment.apps/nginxdeploy created
发现deploy不正常。
root@k8scludes1:~/minsvcbug# kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginxdeploy 0/1 0 0 18s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy
pod也没有生成。
root@k8scludes1:~/minsvcbug# kubectl get pod -o wide
No resources found in minsvcbug namespace.
注意：当我们创建一个deploy，比如nginxdeploy，则系统会为我们创建一个replicaset，名字为nginxdeploy-XXX，我们在deploy里指定的副本，本质上是由replicaset创建出来的，pod名字为nginxdeploy-XXX-yyyy。
可以发现，deployment和replicaset都没有ready。
root@k8scludes1:~/minsvcbug# kubectl get all -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/nginxdeploy 0/1 0 0 7m17s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/nginxdeploy-5d458fcbfc 1 0 0 7m17s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy,pod-template-hash=5d458fcbfc
在kube-system命名空间里有很多个给控制器使用的sa。
root@k8scludes1:~/minsvcbug# kubectl get sa -n kube-system
NAME SECRETS AGE
attachdetach-controller 1 38d
bootstrap-signer 1 38d
calico-kube-controllers 1 38d
calico-node 1 38d
certificate-controller 1 38d
......
persistent-volume-binder 1 38d
pod-garbage-collector 1 38d
pv-protection-controller 1 38d
pvc-protection-controller 1 38d
replicaset-controller 1 38d
replication-controller 1 38d
resourcequota-controller 1 38d
root-ca-cert-publisher 1 38d
service-account-controller 1 38d
service-controller 1 38d
statefulset-controller 1 38d
token-cleaner 1 38d
ttl-after-finished-controller 1 38d
ttl-controller 1 38d
只查看和controller有关的SA。
root@k8scludes1:~/minsvcbug# kubectl get sa -n kube-system | egrep '*controller'
attachdetach-controller 1 38d
calico-kube-controllers 1 38d
certificate-controller 1 38d
clusterrole-aggregation-controller 1 38d
cronjob-controller 1 38d
daemon-set-controller 1 38d
deployment-controller 1 38d
disruption-controller 1 38d
endpoint-controller 1 38d
endpointslice-controller 1 38d
endpointslicemirroring-controller 1 38d
ephemeral-volume-controller 1 38d
expand-controller 1 38d
job-controller 1 38d
namespace-controller 1 38d
node-controller 1 38d
pv-protection-controller 1 38d
pvc-protection-controller 1 38d
replicaset-controller 1 38d
replication-controller 1 38d
resourcequota-controller 1 38d
service-account-controller 1 38d
service-controller 1 38d
statefulset-controller 1 38d
ttl-after-finished-controller 1 38d
ttl-controller 1 38d
控制器创建pod的过程：在kube-system里有很多个给控制器使用的sa,启用PSP之后，现在控制器的SA都访问不了PSP规则，就创建不了pod,只有控制器的sa能访问PSP规则，才能正常创建pod。
现在需要给控制器SA进行授权，使控制器能访问PSP规则。
进行角色绑定，表示kube-system命名空间里的所有serviceaccounts都可以访问psp-allowpri-pod规则。
root@k8scludes1:~/minsvcbug# kubectl get clusterrole | grep cluster
cert-manager-controller-clusterissuers 2022-04-24T17:07:47Z
cluster-admin 2022-04-16T14:57:55Z
clusterole1 2022-05-25T07:31:28Z
clusterrole--psp-allowpri-rule 2022-05-25T08:57:00Z
clusterrole--psp-denypri-rule 2022-05-25T08:09:01Z
system:controller:clusterrole-aggregation-controller 2022-04-16T14:57:55Z
root@k8scludes1:~/minsvcbug# kubectl create rolebinding rolebind-allowpri-psp-kube-system --clusterrole=clusterrole--psp-allowpri-rule --group=system:serviceaccounts -n kube-system --dry-run=client -o yaml > rolebind-allowpri-kube-system.yaml
root@k8scludes1:~/minsvcbug# kubectl apply -f rolebind-allowpri-kube-system.yaml
rolebinding.rbac.authorization.k8s.io/rolebind-allowpri-psp-kube-system created
删除创建失败的nginxdeploy。
root@k8scludes1:~/minsvcbug# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginxdeploy 0/1 0 0 3h13m
root@k8scludes1:~/minsvcbug# kubectl delete deploy nginxdeploy
deployment.apps "nginxdeploy" deleted
root@k8scludes1:~/minsvcbug# kubectl get deployments
No resources found in minsvcbug namespace.
再次创建deploy。
root@k8scludes1:~/minsvcbug# kubectl apply -f nginxdeploy.yaml
deployment.apps/nginxdeploy created
deploy还是没有创建成功。
root@k8scludes1:~/minsvcbug# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginxdeploy 0/1 0 0 8s
root@k8scludes1:~/minsvcbug# kubectl get pod
No resources found in minsvcbug namespace.
查看nginxdeploy描述信息：replicaset-controller Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []，可以发现权限还是不够。
root@k8scludes1:~/minsvcbug# kubectl describe deployment nginxdeploy
Name: nginxdeploy
Namespace: minsvcbug
......
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
ReplicaFailure True FailedCreate
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 34s (x14 over 75s) replicaset-controller Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []
kubectl get ev查看事件。
root@k8scludes1:~/minsvcbug# kubectl get ev
LAST SEEN TYPE REASON OBJECT MESSAGE
11m Warning FailedCreate replicaset/nginxdeploy-5d458fcbfc Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []
101s Warning FailedCreate replicaset/nginxdeploy-5d458fcbfc Error creating: pods "nginxdeploy-5d458fcbfc-" is forbidden: PodSecurityPolicy: unable to admit pod: []
4m25s Normal ScalingReplicaSet deployment/nginxdeploy Scaled up replica set nginxdeploy-5d458fcbfc to 1
删除nginxdeploy。
root@k8scludes1:~/minsvcbug# kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
nginxdeploy 0/1 0 0 5m18s
root@k8scludes1:~/minsvcbug# kubectl delete deploy nginxdeploy
deployment.apps "nginxdeploy" deleted
root@k8scludes1:~/minsvcbug# kubectl get deployments
No resources found in minsvcbug namespace.
root@k8scludes1:~/minsvcbug# kubectl get all
No resources found in minsvcbug namespace.
编辑ClusterRoleBinding配置文件，把RoleBinding变为ClusterRoleBinding，使其全局生效，刚才的角色绑定rolebind-allowpri-psp-kube-system是错误的。
root@k8scludes1:~/minsvcbug# vim rolebind-allowpri-kube-system.yaml
root@k8scludes1:~/minsvcbug# cat rolebind-allowpri-kube-system.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
creationTimestamp: null
name: clusterrolebind-allowpri-psp-kube-system
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: clusterrole--psp-allowpri-rule
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
删除rolebind-allowpri-psp-kube-system。
root@k8scludes1:~/minsvcbug# kubectl get rolebinding -n kube-system | grep rolebind-allowpri-psp-kube-system
rolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 12m
root@k8scludes1:~/minsvcbug# kubectl delete rolebinding rolebind-allowpri-psp-kube-system -n kube-system
rolebinding.rbac.authorization.k8s.io "rolebind-allowpri-psp-kube-system" deleted
root@k8scludes1:~/minsvcbug# kubectl get rolebinding -n kube-system | grep rolebind-allowpri-psp-kube-system
创建clusterrolebinding。
root@k8scludes1:~/minsvcbug# kubectl apply -f rolebind-allowpri-kube-system.yaml
clusterrolebinding.rbac.authorization.k8s.io/clusterrolebind-allowpri-psp-kube-system created
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding -n kube-system | egrep clusterrolebind-allowpri-psp-kube-system
clusterrolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 45s
创建deployment。
root@k8scludes1:~/minsvcbug# kubectl apply -f nginxdeploy.yaml
deployment.apps/nginxdeploy created
现在deploy就创建成功了。
root@k8scludes1:~/minsvcbug# kubectl get all -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/nginxdeploy-5d458fcbfc-rs42b 1/1 Running 0 12s 10.244.218.160 k8scludes2 <none> <none>
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
deployment.apps/nginxdeploy 1/1 1 1 12s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy
NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR
replicaset.apps/nginxdeploy-5d458fcbfc 1 1 1 12s nginx hub.c.163.com/library/nginx:latest app=nginxdeploy,pod-template-hash=5d458fcbfc
root@k8scludes1:~/minsvcbug# kubectl get pod
NAME READY STATUS RESTARTS AGE
nginxdeploy-5d458fcbfc-rs42b 1/1 Running 0 71s
clusterrolebinding全局生效，所有命名空间都能看到。
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding | egrep clusterrolebind-allowpri-psp-kube-system
clusterrolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 4m43s
root@k8scludes1:~/minsvcbug# kubectl get clusterrolebinding -n default | egrep clusterrolebind-allowpri-psp-kube-system
clusterrolebind-allowpri-psp-kube-system ClusterRole/clusterrole--psp-allowpri-rule 4m53s
删除nginxdeploy。
root@k8scludes1:~/minsvcbug# kubectl delete deploy nginxdeploy
deployment.apps "nginxdeploy" deleted
root@k8scludes1:~/minsvcbug# kubectl get all
No resources found in minsvcbug namespace.
十四.总结
PodSecurityPolicy 是 Kubernetes 集群安全的重要组成部分。通过定义 Pod 的安全策略，管理员可以确保集群中的 Pod 遵循特定的安全标准，从而降低潜在的安全风险。本文介绍了 PSP 的概念、重要性以及如何在 Kubernetes 集群中实施。希望这些信息能够帮助您更好地保护您的 Kubernetes 集群。
标签：kubectl,psp,minsvcbug,PodSecurityPolicy,安全策略,Pod,root,pod,k8scludes1 From： https://www.cnblogs.com/renshengdezheli/p/18230999