RBAC授权
1、背景
RBAC: role based access control基于角色的访问控制
# 认证
# 在k8s集群中，对user account认证表示该用户能够作为一个合法的用户去访问apiserver。
# 对service account认证表示pod内能否访问K8s集群资源，同样也是能否正常访问APIserver。
# 授权
# 对被认为是合法的用户进行控制，换言之，该用户能对集群做些什么操作。
# 准入控制
# 准入控制表示对环境资源进行授权分配。
# 客户端访问K8S资源的认证方式主要有两种：客户端认证（利用ca进行双向认证）和bearertoken（利用apiserver提供的密钥进行认证）
# 对Pod内访问k8s资源是通过service account认证完成。service account包括namespace、ca、token等信息。并将上述信息以volume的形似挂载给pod，这样pod就可以携带相应信息与APIserver进行通讯了。
2、service account使用
# 每一个namespace创建时，都会创建一个默认的service account账户。
# 创建一个test-ns的账户namespace
[root@master-worker-node-1 rbac]# kubectl create namespace test-ns
namespace/test-ns created
[root@master-worker-node-1 rbac]# kubectl get ns test-ns
NAME STATUS AGE
test-ns Active 2m54s
# 每个namespace都有一个default的service account账户
[root@master-worker-node-1 rbac]# kubectl get serviceaccount -n test-ns
NAME SECRETS AGE
default 0 3m50s
# 在namespace中创建pod时，不指定pod所用的service account将使用default service account。
[root@master-worker-node-1 rbac]# kubectl get pods -n test-ns -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
test-1 1/1 Running 0 10s 10.244.31.38 only-worker-node-3 <none> <none>
[root@master-worker-node-1 rbac]# kubectl get pods -n test-ns -o yaml | grep serviceAccount -w
serviceAccount: default
# 指定service account
# 先创建一个service account
[root@master-worker-node-1 rbac]# kubectl create serviceaccount -n test-ns test-sa
serviceaccount/test-sa created
[root@master-worker-node-1 rbac]# cat pod-nginx.yaml
apiVersion: v1
kind: Pod
metadata:
name: test-2
namespace: test-ns
spec:
serviceAccount: test-sa
containers:
- name: nginx-2
image: nginx
imagePullPolicy: IfNotPresent
[root@master-worker-node-1 rbac]# kubectl get pods -n test-ns -o yaml test-2 | grep serviceAccount: -w
serviceAccount: test-sa
# 默认没有对service进行授权，因此从pod内部去调用访问apiserver是没办法访问的。
# 实验通过采用RBAC授权方式进行。
3、RBAC授权时的四种资源对象
3.1、RBAC授权时的Role对象
# role的影响范围是某一namespace
Role is a namespaced, logical grouping of PolicyRules that can be referenced as a unit by a RoleBinding.
# role属于k8s中的一类资源，可以通过yaml文件配置
[root@master-worker-node-1 rbac]# kubectl explain role
# 在default 命名空间中的role 具有对pod的读写权限
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: test-role
namespace: default
rules:
- apiGroups: [""] # 标明APIgroup
resources: ["pods"] # 组里面的资源，pods,deployment,replicaset,jobs
verbs: ["get","watch","list"] # 对组里面的资源拥有什么权限。
# 也可以通过命令行创建
[root@master-worker-node-1 rbac]# kubectl create role test-role-2 --resource=pods --verb=get,watch,list --namespace test-ns
role.rbac.authorization.k8s.io/test-role-2 created
[root@master-worker-node-1 rbac]# kubectl get role -n test-ns
NAME CREATED AT
test-role-2 2022-12-31T15:19:38Z
3.2、RBAC授权时的clusterRole对象
# clusterrole是集群的概念
ClusterRole is a cluster level, logical grouping of PolicyRules that can be referenced as a unit by a RoleBinding or ClusterRoleBinding.
[root@master-worker-node-1 rbac]# kubectl explain clusterrole
# k8S中的资源均可以通过yaml创建
[root@master-worker-node-1 rbac]# cat clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: test-cluster-role
# 因为cluster role是集群的概念，所以指定namespace也会被忽略
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get","watch","list"]
# 也可以通过命令行创建
[root@master-worker-node-1 rbac]# kubectl create clusterrole test-cluster-role-2 --verb=get,list,watch --resource=pods
clusterrole.rbac.authorization.k8s.io/test-cluster-role-2 created
[root@master-worker-node-1 rbac]# kubectl get clusterrole | grep cluster-role
test-cluster-role 2022-12-31T15:34:49Z
test-cluster-role-2 2022-12-31T15:36:44Z
3.3、RBAC授权时的RoleBinding对象
# 可以将user通过role binding绑定到一个同命名空间下的role或者全局的cluster role。rolebinding的作用范围仅在给定的namespace中。
RoleBinding references a role, but does not contain it. It can reference a Role in the same namespace or a ClusterRole in the global namespace. It adds who information via Subjects and namespace information by which namespace it exists in. RoleBindings in a given namespace only have effect in that namespace.
# 创建rolebinding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: test-rolebinding
namespace: test-ns # 此处决定了role binding生效的命名空间。
roleRef: # 将rolebinding与哪一个role或者clusterrole进行绑定
apiGroup: rbac.authorization.k8s.io
kind: Role # 只能是Role或者ClusterRole
name: test-role-2
subjects: # 将rolebinding应用与哪一个user，service account or user？
- kind: ServiceAccount # ServiceAccount or User or Group
name: test-sa
apiGroup: rbac.authorization.k8s.io
[root@master-worker-node-1 rbac]# kubectl apply -f rolebinding.yaml
rolebinding.rbac.authorization.k8s.io/test-rolebinding created
[root@master-worker-node-1 rbac]# kubectl get rolebinding -n test-ns
NAME ROLE AGE
test-rolebinding Role/test-role-2 80s
[root@master-worker-node-1 rbac]# kubectl describe rolebinding -n test-ns
Name: test-rolebinding
Labels: <none>
Annotations: <none>
Role:
Kind: Role
Name: test-role-2
Subjects:
Kind Name Namespace
---- ---- ---------
ServiceAccount test-sa
# 通过kubectl命令行也是可以创建的
# 创建一个名称为test-rolebinding-2的rolebinding，它将user-test-rolebinding-user-2用户和clusterrole test-cluster-role-2进行绑定，可操作的命名空间为test-ns.
[root@master-worker-node-1 rbac]# kubectl create rolebinding test-rolebinding-2 --user=test-rolebinding-user-2 --clusterrole test-cluster-role-2 --namespace test-ns
rolebinding.rbac.authorization.k8s.io/test-rolebinding-2 created
[root@master-worker-node-1 rbac]# kubectl get rolebinding -n test-ns
NAME ROLE AGE
test-rolebinding Role/test-role-2 8m43s
test-rolebinding-2 ClusterRole/test-cluster-role-2 2m1s
3.4、RBAC授权时的ClusterRoleBinding对象
ClusterRoleBinding references a ClusterRole, but not contain it. It can reference a ClusterRole in the global namespace, and adds who information via Subject.
[root@master-worker-node-1 rbac]# cat clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: test-clusterrolebinding-1
subjects: # 确定clusterrolebinding作用于哪一个user
- kind: User # User or Group or ServiceAccount
name: test-clusterolebinding-user-1
apiGroup: rbac.authorization.k8s.io
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: test-cluster-role
[root@master-worker-node-1 rbac]# kubectl apply -f clusterrolebinding.yaml
clusterrolebinding.rbac.authorization.k8s.io/test-clusterrolebinding-1 created
[root@master-worker-node-1 rbac]# kubectl get clusterrolebinding | grep -E "NAME|test"
NAME ROLE AGE
test-clusterrolebinding-1 ClusterRole/test-cluster-role 39s
[root@master-worker-node-1 rbac]# kubectl describe clusterrolebinding test-clusterrolebinding-1
Name: test-clusterrolebinding-1
Labels: <none>
Annotations: <none>
Role:
Kind: ClusterRole
Name: test-cluster-role
Subjects:
Kind Name Namespace
---- ---- ---------
User test-clusterolebinding-user-1
# 通过kubectl命令行创建
# 创建一个名字为test-clusterrolebinding-2的clueterrolebinding，并将serviceaccount test:test-sa与clusterrole进行绑定。
[root@master-worker-node-1 rbac]# kubectl create clusterrolebinding test-clusterrolebinding-2 --clusterrole test-cluster-role-2 --serviceaccount test-ns:test-sa
clusterrolebinding.rbac.authorization.k8s.io/test-clusterrolebinding-2 created
[root@master-worker-node-1 rbac]# kubectl get clusterrolebinding | grep -E "NAME|test"
NAME ROLE AGE
test-clusterrolebinding-1 ClusterRole/test-cluster-role 6m3s
test-clusterrolebinding-2 ClusterRole/test-cluster-role-2 119s
4、创建一个user account账户控制K8S集群
# 根据kubectl config view 可以查看到当前使用kubectl命令是所使用的用户信息
[root@master-worker-node-1 rbac]# kubectl config view apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://192.168.122.253:16443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin # 当前用户
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
# 这里因为当前主机上的/root/.kube/config目录中指定了上述相应的信息。
[root@master-worker-node-1 rbac]# ls -l /root/.kube/config
-rw-------. 1 root root 5644 Nov 25 12:45 /root/.kube/config
[root@master-worker-node-1 rbac]# cat ~/.kube/config | grep user
user: kubernetes-admin
users:
user:
# 如果我将/root/.kube/config文件拷贝到worker node，那么在worker node同样可以实现通过Kubernetes-admin账户对集群进行管理。
# 此处测试新增一个用户进行k8s集群管理
# 此处采用ssl认证
# 1、生成私钥
[root@master-worker-node-1 rbac]# openssl genrsa -out /etc/kubernetes/pki/test.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
...............................................+++++
.............+++++
e is 65537 (0x010001)
# 2、生成证书请求
[root@master-worker-node-1 rbac]# openssl req -new -key /etc/kubernetes/pki/test.key -out /etc/kubernetes/pki/test.csr -subj "/CN=test"
# 3、签发证书
[root@master-worker-node-1 rbac]# openssl x509 -req -in /etc/kubernetes/pki/test.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out /etc/kubernetes/pki/test.crt -days 3650
Signature ok
subject=CN = test
Getting CA Private Key
# 4、K8S集群新增一个test用户，以后就用test用户对apiserver进行认证，连接
[root@master-worker-node-1 rbac]# kubectl config set-credentials test --client-certificate=/etc/kubernetes/pki/test.crt --client-key=/etc/kubernetes/pki/test.key --embed-certs=true
User "test" set.
# 此时通过kubectl config view就可以看到新增的test用户
[root@master-worker-node-1 rbac]# kubectl config view | tail
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: test
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
[root@master-worker-node-1 rbac]# kubectl config get-users
NAME
kubernetes-admin
test
# 5、将test用户设置为可以控制对应K8S集群的账号
[root@master-worker-node-1 rbac]# kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@kubernetes kubernetes kubernetes-admin
[root@master-worker-node-1 rbac]# kubectl config set-context test@kubernetes --cluster=kubernetes --user=test
Context "test@kubernetes" created.
[root@master-worker-node-1 rbac]# kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@kubernetes kubernetes kubernetes-admin
test@kubernetes kubernetes test
# 6、将控制kubernetes这个集群的上下文context切换为test
# 当前名为kubernetes的集群使用的context为kubernetes-admin，这个是集群账号，有所有的权限
[root@master-worker-node-1 rbac]# kubectl config get-clusters
NAME
kubernetes
[root@master-worker-node-1 rbac]# kubectl config view | grep current
current-context: kubernetes-admin@kubernetes
# 将context切换到test
[root@master-worker-node-1 rbac]# kubectl config use-context test@kubernetes
Switched to context "test@kubernetes".
[root@master-worker-node-1 rbac]# kubectl config view | grep current
current-context: test@kubernetes
# 创建的普通用户默认是没有任何权限的，需要先对test用户进行rbac授权
[root@master-worker-node-1 rbac]# kubectl get nodes
Error from server (Forbidden): nodes is forbidden: User "test" cannot list resource "nodes" in API group "" at the cluster scope
# 6、对test用户进行RBAC授权
# 需要切换到kubernetes进行授权
[root@master-worker-node-1 rbac]# kubectl config use-context kubernetes-admin@kubernetes
Switched to context "kubernetes-admin@kubernetes".
# 此处采用rolebinding绑定到cluster-role，只允许test用户在test这个namespace进行操作
[root@master-worker-node-1 rbac]# kubectl create ns test
namespace/test created
[root@master-worker-node-1 rbac]# kubectl create rolebinding test --namespace test --clusterrole cluster-admin --user test
rolebinding.rbac.authorization.k8s.io/test created
# 将context切换到test@kubernetes测试
[root@master-worker-node-1 rbac]# kubectl config use-context test@kubernetes
Switched to context "test@kubernetes".
# 查看集群资源是不允许的，因为rolebinding只允许查看对应namespace中的资源
[root@master-worker-node-1 rbac]# kubectl get nodes
Error from server (Forbidden): nodes is forbidden: User "test" cannot list resource "nodes" in API group "" at the cluster scope
[root@master-worker-node-1 rbac]# kubectl get pods -o wide -n test
No resources found in test namespace.
# 在test namespace中创建deployment资源，可以正常查看test namespace中的资源。
[root@master-worker-node-1 rbac]# kubectl get all -n test
NAME READY STATUS RESTARTS AGE
pod/test-context-86d87d7778-9c2kg 1/1 Running 0 40s
pod/test-context-86d87d7778-c97ww 1/1 Running 0 40s
pod/test-context-86d87d7778-z24wq 1/1 Running 0 40s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/test-context 3/3 3 3 41s
NAME DESIRED CURRENT READY AGE
replicaset.apps/test-context-86d87d7778 3 3 3 41s
5、创建一个user account账户可以查看所有的pods和nodes
# 此处正常使用之前创建的test用户
# 创建一个clusterrole对所有pods和nodes进行查看
[root@master-worker-node-1 ~]# kubectl create clusterrole test --verb=get,watch,list --resource=pods,nodes
clusterrole.rbac.authorization.k8s.io/test created
# 创建clusterrolebinding将user和clusterrole进行关联
[root@master-worker-node-1 ~]# kubectl create clusterrolebinding test --clusterrole=test --user=test
clusterrolebinding.rbac.authorization.k8s.io/test created
# 切换context，发现确实只能查看pods和nodes，不能看到其他资源
[root@master-worker-node-1 ~]# kubectl config use-context test@kubernetes
Switched to context "test@kubernetes".
[root@master-worker-node-1 ~]#
[root@master-worker-node-1 ~]# kubectl get deployment -A
Error from server (Forbidden): deployments.apps is forbidden: User "test" cannot list resource "deployments" in API group "apps" at the cluster scope
[root@master-worker-node-1 ~]# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
default mynginx-ff886775c-kkjp9 1/1 Running 0 2d5h
default mynginx-ff886775c-lcwcx 1/1 Running 0 2d10h
default nfs-provisioner-9f9fc45fd-l4f6l 1/1 Running 2 (11d ago) 12d
kube-system calico-kube-controllers-6744f6b6d5-wqk7s 1/1 Running 0 36d
kube-system calico-node-49qt2 1/1 Running 0 41d
kube-system calico-node-c28bx 1/1 Running 0 41d
kube-system calico-node-q2wpg 1/1 Running 0 41d
kube-system calico-node-tpxlp 1/1 Running 0 41d
kube-system coredns-7f8cbcb969-rtrdx 1/1 Running 0 36d
kube-system coredns-7f8cbcb969-x7nlz 1/1 Running 0 36d
kube-system etcd-master-worker-node-1 1/1 Running 5 41d
kube-system etcd-master-worker-node-2 1/1 Running 0 41d
kube-system kube-apiserver-master-worker-node-1 1/1 Running 32 41d
kube-system kube-apiserver-master-worker-node-2 1/1 Running 1 (41d ago) 41d
kube-system kube-controller-manager-master-worker-node-1 1/1 Running 13 (11d ago) 41d
kube-system kube-controller-manager-master-worker-node-2 1/1 Running 3 (11d ago) 41d
kube-system kube-proxy-78hcj 1/1 Running 0 41d
kube-system kube-proxy-7gjz9 1/1 Running 0 41d
kube-system kube-proxy-c4d2m 1/1 Running 0 41d
kube-system kube-proxy-nqdzd 1/1 Running 0 41d
kube-system kube-scheduler-master-worker-node-1 1/1 Running 10 (11d ago) 41d
kube-system kube-scheduler-master-worker-node-2 1/1 Running 3 (11d ago) 41d
kube-system metrics-server-548f77b868-7qj45 1/1 Running 0 2d
test-ns test-1 1/1 Running 0 4d23h
test-ns test-2 1/1 Running 0 4d23h
test test-context-86d87d7778-brsxv 1/1 Running 1 (2m45s ago) 21m
test test-context-86d87d7778-rx7n5 1/1 Running 1 (2m44s ago) 21m
test test-context-86d87d7778-zhwdn 1/1 Running 1 (2m44s ago) 21m
[root@master-worker-node-1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master-worker-node-1 Ready control-plane 41d v1.25.3
master-worker-node-2 Ready control-plane 41d v1.25.3
only-worker-node-3 Ready worker 41d v1.25.3
only-worker-node-4 Ready worker 41d v1.25.3
6、小结
1、RBAC授权中的主要资源有4种：role、clusterrole、rolebinding、clusterrolebinding，它们一共可以组合出三种关系。
2、role和roleBinding都有namespace限制，即只能在相应的namespace下操作