持久卷PV
官网文档
https://kubernetes.io/zh-cn/docs/concepts/storage/persistent-volumes/
什么是PV和PVC
持久卷（PersistentVolume，PV） 是集群中的一块存储，可以由管理员事先制备， 或者使用存储类（Storage Class）来动态制备。 持久卷是集群资源，就像节点也是集群资源一样。PV 持久卷和普通的 Volume 一样， 也是使用卷插件来实现的，只是它们拥有独立于任何使用 PV 的 Pod 的生命周期。 此 API 对象中记述了存储的实现细节，无论其背后是 NFS、iSCSI 还是特定于云平台的存储系统。
持久卷申领（PersistentVolumeClaim，PVC） 表达的是用户对存储的请求。概念上与 Pod 类似。 Pod 会耗用节点资源，而 PVC 申领会耗用 PV 资源。Pod 可以请求特定数量的资源（CPU 和内存）；同样 PVC 申领也可以请求特定的大小和访问模式 （例如，可以要求 PV 卷能够以 ReadWriteOnce、ReadOnlyMany 或 ReadWriteMany 模式之一来挂载，参见访问模式）。
尽管 PersistentVolumeClaim 允许用户消耗抽象的存储资源， 常见的情况是针对不同的问题用户需要的是具有不同属性（如，性能）的 PersistentVolume 卷。 集群管理员需要能够提供不同性质的 PersistentVolume， 并且这些 PV 卷之间的差别不仅限于卷大小和访问模式，同时又不能将卷是如何实现的这些细节暴露给用户。 为了满足这类需求，就有了存储类（StorageClass） 资源。
总结：PV是一种集群资源，PVC是对这类集群资源的请求。
PV的制作
制作分为两种：动态、静态
PV的类型
PV 持久卷是用插件的形式来实现的。Kubernetes 目前支持以下插件：
cephfs - CephFS volume
csi - 容器存储接口 (CSI)
fc - Fibre Channel (FC) 存储
hostPath - HostPath 卷 （仅供单节点测试使用；不适用于多节点集群；请尝试使用 local 卷作为替代）
iscsi - iSCSI (SCSI over IP) 存储
local - 节点上挂载的本地存储设备
nfs - 网络文件系统 (NFS) 存储
rbd - Rados 块设备 (RBD) 卷
PV的访问模式
在命令行接口（CLI）中，访问模式也使用以下缩写形式：
RWO - ReadWriteOnce
ROX - ReadOnlyMany
RWX - ReadWriteMany
RWOP - ReadWriteOncePod
创建PV
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-1
spec:
accessModes:
- ReadWriteMany
capacity:
storage: 100Mi
nfs:
path: /mnt/dir-1
server: 192.168.122.106
[root@master-worker-node-1 storage]# kubectl get persistentvolume -o wide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
pv-1 100Mi RWX Retain Available 7m3s Filesystem
pv-2 200Mi RWX Retain Available 24s Filesystem
pv-3 300Mi RWX Retain Available 22s Filesystem
pv-4 400Mi RWX Retain Available 19s Filesystem
pv-5 500Mi RWX Retain Available 16s Filesystem
创建PVC
# 创建PVC可以通过selector指定PV，或者PV范围，也可以通过volumeName指定特定的PV，也可以通过resources大小自动完成匹配。
apiVersion: vi
kind: PersistenVolumeClaim
metadata:
name: pvc-1
spec:
accessModes:
- ReadWriteMany
volumeName: pv-1 # 静态绑定可以通过指定PV的名称
#selector: # 也可以通过指定PV的labels
# matchLabels:
# sssss:sssss
resources:
requests:
storage: 100Mi
# 通过名称进行绑定
[root@master-worker-node-1 storage]# kubectl apply -f pvc.yaml
persistentvolumeclaim/pvc-1 created
persistentvolumeclaim/pvc-2 created
persistentvolumeclaim/pvc-3 created
# 通过resources大小进行绑定
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-450
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 450Mi
[root@master-worker-node-1 storage]# kubectl get pvc -o wide
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE
pvc-1 Bound pv-1 100Mi RWX 2m31s Filesystem
pvc-2 Bound pv-2 200Mi RWX 2m31s Filesystem
pvc-3 Bound pv-3 300Mi RWX 2m31s Filesystem
pvc-450 Bound pv-5 500Mi RWX 7s Filesystem
pod管理volume
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-pv
spec:
selector:
matchLabels:
name: test-pvc-1
replicas: 3
template:
metadata:
labels:
name: test-pvc-1
spec:
containers:
- name: nginx
image: nginx:stable-alpine-perl
imagePullPolicy: IfNotPresent
volumeMounts:
- name: pvc-1
mountPath: /usr/share/nginx/html
volumes:
- persistentVolumeClaim:
claimName: pvc-1
name: pvc-1
[root@master-worker-node-1 storage]# kubectl expose deployment test-pv --type='ClusterIP' --target-port='80' --port='30080'
service/test-pv exposed
[root@master-worker-node-1 storage]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
test-pv-dfc7ff6b8-crqrz 1/1 Running 0 4m57s 10.244.54.4 only-worker-node-4 <none> <none>
test-pv-dfc7ff6b8-rsk9j 1/1 Running 0 4m57s 10.244.31.18 only-worker-node-3 <none> <none>
test-pv-dfc7ff6b8-zzq6k 1/1 Running 0 4m57s 10.244.31.17 only-worker-node-3 <none> <none>
[root@master-worker-node-1 storage]# kubectl get deployment -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
test-pv 3/3 3 3 5m6s nginx nginx:stable-alpine-perl name=test-pvc-1
[root@master-worker-node-1 storage]# kubectl get service -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 28d <none>
test-pv ClusterIP 10.108.129.82 <none> 30080/TCP 94s name=test-pvc-1
[root@master-worker-node-1 storage]# curl 10.108.129.82:30080
dir-1
测试PVC的删除和PV的回收策略
要想删除PVC需要先删除使用PVC的pod
[root@master-worker-node-1 storage]# kubectl delete -f deploy.yaml
deployment.apps "test-pv" deleted
# 默认PV的回收策略是retain，保留
# 删除PV
[root@master-worker-node-1 storage]# kubectl delete pvc pvc-1
persistentvolumeclaim "pvc-1" deleted
# 此时之前绑定的PV处于release状态
[root@master-worker-node-1 storage]# kubectl get pv pv-1
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv-1 100Mi RWX Retain Released default/pvc-1 100m
# 此时保存在PV 中的文件不受影响
[root@master-worker-node-1 storage]# ssh node2 ls -l /mnt/dir-1
total 4
-rw-r--r--. 1 root root 6 Dec 23 13:39 index.html
# 因为PV与PVC是一对一的关系，刚才删除了对应的PVC，如果想要再次使用这个PV，直接进行绑定是不行的
[root@master-worker-node-1 storage]# cat second-time-pvc-1.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-1
spec:
accessModes:
- ReadWriteMany
volumeName: pv-1
#selector:
# matchLabels:
# sssss:sssss
resources:
requests:
storage: 100Mi
[root@master-worker-node-1 storage]# kubectl apply -f second-time-pvc-1.yaml
persistentvolumeclaim/pvc-1 created
# 发现PVC的状态是PENDING
[root@master-worker-node-1 storage]# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-1 Pending pv-1 0 111s
pvc-2 Bound pv-2 200Mi RWX 35m
pvc-3 Bound pv-3 300Mi RWX 35m
pvc-450 Bound pv-5 500Mi RWX 32m
# 查看详情，提示该PV已经绑定给了其他PVC
[root@master-worker-node-1 storage]# kubectl describe pvc pvc-1
Name: pvc-1
Namespace: default
StorageClass:
Status: Pending
Volume: pv-1
Labels: <none>
Annotations: <none>
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 0
Access Modes:
VolumeMode: Filesystem
Used By: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedBinding 10s (x10 over 2m25s) persistentvolume-controller volume "pv-1" already bound to a different claim.
# 这是因为，PV侧的绑定记录并不随着PVC的删除而删除
[root@master-worker-node-1 storage]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv-1 100Mi RWX Retain Released default/pvc-1 107m
# 对于这种情况，需要先删除一下PV
[root@master-worker-node-1 storage]# kubectl delete pvc pvc-1
persistentvolumeclaim "pvc-1" deleted
[root@master-worker-node-1 storage]# kubectl delete pv pv-1
persistentvolume "pv-1" deleted
[root@master-worker-node-1 storage]# kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv-2 200Mi RWX Retain Bound default/pvc-2 102m
pv-3 300Mi RWX Retain Bound default/pvc-3 102m
pv-4 400Mi RWX Retain Available 102m
pv-5 500Mi RWX Retain Bound default/pvc-450 102m
# 删除PV，PV的回收策略是retain，文件也不受影响
[root@master-worker-node-1 storage]# ssh node2 ls -l /mnt/dir-1
total 4
-rw-r--r--. 1 root root 6 Dec 23 13:39 index.html
# 要想继续使用这个PV，需要重新创建
[root@master-worker-node-1 storage]# kubectl apply -f pv-1.yaml
persistentvolume/pv-1 created
[root@master-worker-node-1 storage]# kubectl get pv -o wide
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE VOLUMEMODE
pv-1 100Mi RWX Retain Available 40s Filesystem
pv-2 200Mi RWX Retain Bound default/pvc-2 104m Filesystem
pv-3 300Mi RWX Retain Bound default/pvc-3 104m Filesystem
pv-4 400Mi RWX Retain Available 104m Filesystem
pv-5 500Mi RWX Retain Bound default/pvc-450 104m Filesystem
# 此时PVC就可以正常申请和PV进行绑定
[root@master-worker-node-1 storage]# kubectl apply -f second-time-pvc-1.yaml
persistentvolumeclaim/pvc-1 created
[root@master-worker-node-1 storage]# kubectl get pv pv-1
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv-1 100Mi RWX Retain Bound default/pvc-1 96s
[root@master-worker-node-1 storage]# kubectl get pvc pvc-1
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pvc-1 Bound pv-1 100Mi RWX 20s
小结
1、PV是一种集群存储资源，PVC是对该资源的请求
2、PV支持多种类型的后端存储，也支持多种接入模式accessmode（RWO、ROX、RWX）
3、PV和PVC是一一绑定的关系，当PV和PVC进行绑定以后，删除PVC后，PV是不能直接再次申请的，需要先删除。
4、PVC的使用大致分为几步：创建PV、创建PVC、POD进行绑定；为了简化，可以使用storageClass进行动态创建，而不用手动创建PV