一、StatefulSet介绍
官网介绍：https://kubernetes.io/zh/docs/concepts/workloads/controllers/statefulset/
- StatefulSet 是用来管理有状态应用的工作负载 API 对象。
- StatefulSet 用来管理某 Pod 集合的部署和扩缩， 并为这些 Pod 提供持久存储和持久标识符。和 Deployment 类似， StatefulSet 管理基于相同容器规约的一组 Pod。但和 Deployment 不同的是， StatefulSet 为它们的每个 Pod 维护了一个有粘性的 ID。这些 Pod 是基于相同的规约来创建的， 但是不能相互替换：无论怎么调度，每个 Pod 都有一个永久不变的 ID。
- 如果希望使用存储卷为工作负载提供持久存储，可以使用 StatefulSet 作为解决方案的一部分。 尽管 StatefulSet 中的单个 Pod 仍可能出现故障， 但持久的 Pod 标识符使得将现有卷与替换已失败 Pod 的新 Pod 相匹配变得更加容易。
使用 StatefulSet
StatefulSet 对于需要满足以下一个或多个需求的应用程序很有价值：
- 稳定的、唯一的网络标识符。
- 稳定的、持久的存储。
- 有序的、优雅的部署和扩缩。
- 有序的、自动的滚动更新。
在上面描述中，“稳定的”意味着 Pod 调度或重调度的整个过程是有持久性的。 如果应用程序不需要任何稳定的标识符或有序的部署、删除或扩缩， 则应该使用由一组无状态的副本控制器提供的工作负载来部署应用程序，比如 Deployment 或者 ReplicaSet 可能更适用于你的无状态应用部署需要。
限制
- 给定 Pod 的存储必须由 PersistentVolume Provisioner 基于所请求的 storage class 来制备，或者由管理员预先制备。
- 删除或者扩缩 StatefulSet 并不会删除它关联的存储卷。 这样做是为了保证数据安全，它通常比自动清除 StatefulSet 所有相关的资源更有价值。
- StatefulSet 当前需要无头服务来负责 Pod 的网络标识。你需要负责创建此服务。
- 当删除一个 StatefulSet 时，该 StatefulSet 不提供任何终止 Pod 的保证。 为了实现 StatefulSet 中的 Pod 可以有序且体面地终止，可以在删除之前将 StatefulSet 缩容到 0。
- 在默认 Pod 管理策略(OrderedReady) 时使用滚动更新， 可能进入需要人工干预才能修复的损坏状态。
使用场景
- Statefulset为了解决有状态服务的集群部署、 集群之间的数据同步问题(MySQL主从等)
- Statefulset所管理的Pod拥有唯一且固定的Pod名称
- Statefulset按照顺序对pod进行启停、 伸缩和回收
- Headless Services(无头服务， 请求的解析直接解析到pod IP)
运行逻辑
二、基于StatefulSet部署有状态服务
2.1、准备nfs 动态存储类
root@easzlab-deploy:~/jiege-k8s/nfs-pv-pvc# cat nfs-deployment.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["endpoints","services"]
verbs: ["get", "list", "watch","create","update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
namespace: kube-system
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.io
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: nfs-provisioner-01
namespace: kube-system
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-provisioner-01
template:
metadata:
labels:
app: nfs-provisioner-01
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: registry.cn-qingdao.aliyuncs.com/zhangshijie/nfs-subdir-external-provisioner:v4.0.2
imagePullPolicy: IfNotPresent
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: nfs-provisioner-01
- name: NFS_SERVER
value: 172.16.88.169
- name: NFS_PATH
value: /data
volumes:
- name: nfs-client-root
nfs:
server: 172.16.88.169
path: /data
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs-dynamic-class
annotations:
storageclass.kubernetes.io/is-default-class: "false"
provisioner: nfs-provisioner-01
parameters:
archiveOnDelete: "false"
root@easzlab-deploy:~/jiege-k8s/nfs-pv-pvc#
root@easzlab-deploy:~/jiege-k8s/nfs-pv-pvc# kubectl apply -f nfs-deployment.yaml
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
deployment.apps/nfs-provisioner-01 created
storageclass.storage.k8s.io/nfs-dynamic-class created
root@easzlab-deploy:~/jiege-k8s/nfs-pv-pvc# kubectl get pod -n kube-system
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-5c8bb696bb-5nrbl 1/1 Running 3 (2d18h ago) 3d23h
kube-system calico-node-68k76 1/1 Running 2 (2d18h ago) 3d21h
kube-system calico-node-bz8pp 1/1 Running 2 (2d18h ago) 3d21h
kube-system calico-node-cfl2g 1/1 Running 2 (2d18h ago) 3d21h
kube-system calico-node-hsrfh 1/1 Running 2 (2d18h ago) 3d21h
kube-system calico-node-j8kgf 1/1 Running 2 (2d18h ago) 3d21h
kube-system calico-node-kdb5j 1/1 Running 2 (2d18h ago) 3d21h
kube-system calico-node-mqmhp 1/1 Running 2 (2d18h ago) 3d21h
kube-system calico-node-x6ctf 1/1 Running 2 (2d18h ago) 3d21h
kube-system calico-node-xh79g 1/1 Running 2 3d21h
kube-system coredns-69548bdd5f-9md6c 1/1 Running 2 (2d18h ago) 3d23h
kube-system coredns-69548bdd5f-n6rvg 1/1 Running 2 (2d18h ago) 3d23h
kube-system kuboard-7dc6ffdd7c-njhnb 1/1 Running 2 (2d18h ago) 3d23h
kube-system metrics-server-686ff776cf-dbgq6 1/1 Running 4 (2d18h ago) 3d18h
kube-system nfs-provisioner-01-599b7bfc9d-ldl99 1/1 Running 0 3s
kubernetes-dashboard dashboard-metrics-scraper-8c47d4b5d-5cxhb 1/1 Running 2 (2d18h ago) 3d19h
kubernetes-dashboard kubernetes-dashboard-f778f96c9-s5dk7 1/1 Running 4 (2d18h ago) 3d16h
velero-system velero-858b9459f9-4jbns 1/1 Running 4 (2d18h ago) 3d20h
root@easzlab-deploy:~/jiege-k8s/nfs-pv-pvc# kubectl get sc -A
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
ceph-storage-class-k8s (default) kubernetes.io/rbd Delete Immediate true 3d19h
nfs-dynamic-class nfs-provisioner-01 Delete Immediate false 10s
root@easzlab-deploy:~/jiege-k8s/nfs-pv-pvc#
2.2、准备有状态部署文件
官网案例:https://kubernetes.io/zh-cn/docs/concepts/workloads/controllers/statefulset/
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7# vi StatefulSet-nginx.yaml
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7# cat StatefulSet-nginx.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
selector:
matchLabels:
app: nginx
serviceName: "nginx"
replicas: 3
minReadySeconds: 10
template:
metadata:
labels:
app: nginx
spec:
terminationGracePeriodSeconds: 10
containers:
- name: nginx
image: registry.cn-shenzhen.aliyuncs.com/cyh01/nginx-slim:0.8 #这里镜像在国外，只能先想办法上传到自己私仓
ports:
- containerPort: 80
name: web
volumeMounts:
- name: www
mountPath: /usr/share/nginx/html
volumeClaimTemplates:
- metadata:
name: www
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "nfs-dynamic-class"
resources:
requests:
storage: 10Gi
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7#
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7# kubectl apply -f StatefulSet-nginx.yaml
service/nginx created
statefulset.apps/web created
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7#
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7#
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7# kubectl get statefulset -A NAMESPACE NAME READY AGE argocd argocd-application-controller 1/1 3d18h argocd argocd-redis-ha-server 3/3 3d18h default web 3/3 18m root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7# root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7# kubectl get pod -owide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES web-0 1/1 Running 0 78s 10.200.104.238 172.16.88.163 <none> <none> web-1 1/1 Running 0 38s 10.200.105.166 172.16.88.164 <none> <none> root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 3d23h nginx ClusterIP None <none> 80/TCP 87s root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7# kubectl get pvc -A NAMESPACE NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE default www-web-0 Bound pvc-f981dab0-b708-49b9-8b36-a6700cbefa96 10Gi RWO nfs-dynamic-class 3m51s default www-web-1 Bound pvc-d3ccbf91-3541-4b33-b7fc-3f54e27f2759 10Gi RWO nfs-dynamic-class 3m11s default www-web-2 Bound pvc-e5c7d9f2-2808-4086-bc7c-3aefd6c75012 10Gi RWO nfs-dynamic-class 2m31s root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7# kubectl get pv -A NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-d3ccbf91-3541-4b33-b7fc-3f54e27f2759 10Gi RWO Delete Bound default/www-web-1 nfs-dynamic-class 3m15s pvc-e5c7d9f2-2808-4086-bc7c-3aefd6c75012 10Gi RWO Delete Bound default/www-web-2 nfs-dynamic-class 2m34s pvc-f981dab0-b708-49b9-8b36-a6700cbefa96 10Gi RWO Delete Bound default/www-web-0 nfs-dynamic-class 3m55s root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case7#
上述例子中：
- 名为
nginx的 Headless Service 用来控制网络域名。 - 名为
web的 StatefulSet 有一个 Spec，它表明将在独立的 3 个 Pod 副本中启动 nginx 容器。 volumeClaimTemplates将通过 PersistentVolume 制备程序所准备的 PersistentVolumes 来提供稳定的存储。