一、简介
- 首先需要创建一个nfs 服务器，作为存储服务器；
- 将nfs服务器上导出 (export)的文件系统用作存储卷；
- nfs是文件系统级共享服务，它支持多路挂载请求，可由多个Pod对象同时用作存储卷后端；
二、准备NFS服务器
1、安装NFS-server
root@k8s-node03:~# apt search ^nfs-ker
Sorting... Done
Full Text Search... Done
nfs-kernel-server/jammy-updates 1:2.6.1-1ubuntu1.2 amd64
support for NFS kernel server
root@k8s-node03:~# apt install nfs-kernel-server
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
nfs-kernel-server
0 upgraded, 1 newly installed, 0 to remove and 65 not upgraded.
Need to get 140 kB of archives.
After this operation, 526 kB of additional disk space will be used.
Get:1 http://nova.clouds.archive.ubuntu.com/ubuntu jammy-updates/main amd64 nfs-kernel-server amd64 1:2.6.1-1ubuntu1.2 [140 kB]
Fetched 140 kB in 2s (72.6 kB/s)
Selecting previously unselected package nfs-kernel-server.
(Reading database ... 94308 files and directories currently installed.)
Preparing to unpack .../nfs-kernel-server_1%3a2.6.1-1ubuntu1.2_amd64.deb ...
Unpacking nfs-kernel-server (1:2.6.1-1ubuntu1.2) ...
Setting up nfs-kernel-server (1:2.6.1-1ubuntu1.2) ...
Created symlink /etc/systemd/system/nfs-client.target.wants/nfs-blkmap.service → /lib/systemd/system/nfs-blkmap.service.
Created symlink /etc/systemd/system/multi-user.target.wants/nfs-server.service → /lib/systemd/system/nfs-server.service.
nfs-mountd.service is a disabled or a static unit, not starting it.
nfsdcld.service is a disabled or a static unit, not starting it.
Creating config file /etc/exports with new version
Creating config file /etc/default/nfs-kernel-server with new version
Processing triggers for man-db (2.10.2-1) ...
Scanning processes...
Scanning candidates...
Scanning linux images...
Restarting services...
Service restarts being deferred:
/etc/needrestart/restart.d/dbus.service
systemctl restart docker.service
systemctl restart [email protected]
systemctl restart networkd-dispatcher.service
systemctl restart systemd-logind.service
systemctl restart unattended-upgrades.service
No containers need to be restarted.
No user sessions are running outdated binaries.
No VM guests are running outdated hypervisor (qemu) binaries on this host.
2、准备共享目录
root@k8s-node03:~# mkdir -pv /data/redis
mkdir: created directory '/data'
mkdir: created directory '/data/redis'
root@k8s-node03:~# vi /etc/exports
root@k8s-node03:~# cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
/data/redis 192.168.10.0/24(rw,no_root_squash,no_subtree_check)
3、启动NFS服务
root@k8s-node03:~# systemctl restart nfs-server
root@k8s-node03:~# systemctl status nfs-server
● nfs-server.service - NFS server and services
Loaded: loaded (/lib/systemd/system/nfs-server.service; enabled; vendor preset: enabled)
Active: active (exited) since Tue 2024-01-23 22:14:32 CST; 10s ago
Process: 1595248 ExecStartPre=/usr/sbin/exportfs -r (code=exited, status=0/SUCCESS)
Process: 1595250 ExecStart=/usr/sbin/rpc.nfsd (code=exited, status=0/SUCCESS)
Main PID: 1595250 (code=exited, status=0/SUCCESS)
CPU: 10ms
Jan 23 22:14:32 k8s-node03 systemd[1]: Starting NFS server and services...
Jan 23 22:14:32 k8s-node03 systemd[1]: Finished NFS server and services.
三、客户端安装NFS工具
1、安装NFS客户端工具
root@k8s-node02:~# apt install nfs-common
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
nfs-common is already the newest version (1:2.6.1-1ubuntu1.2).
0 upgraded, 0 newly installed, 0 to remove and 65 not upgraded.
root@k8s-node03:~#
2、验证挂载NFS共享目录
root@k8s-node02:~# mount -t nfs 192.168.10.13:/data/redis /mnt/
root@k8s-node02:~# cd /mnt/
root@k8s-node02:/mnt# ls
root@k8s-node02:/mnt# touch test
root@k8s-node02:/mnt# echo 666> test
3、NFS查看已经有数据
root@k8s-node03:~# cat /data/redis/test
666
root@k8s-node03:~#
四、Pod挂载NFS存储卷，验证数据存储的持久性
1、创建Pod并挂载NFS存储卷
root@k8s-master01:~/learning-k8s/examples/volumes# cat pod-with-nfs-vol.yaml
apiVersion: v1
kind: Pod
metadata:
name: redis-nfs-002
spec:
containers:
- name: redis
image: redis:7-alpine
imagePullPolicy: IfNotPresent
volumeMounts:
- name: redisdata
mountPath: /data
volumes:
- name: redisdata
nfs:
server: 192.168.10.13
path: /data/redis
root@k8s-master01:~/learning-k8s/examples/volumes# kubectl apply -f pod-with-nfs-vol.yaml
root@k8s-master01:~/learning-k8s/examples/volumes# kubectl get pod
NAME READY STATUS RESTARTS AGE
redis-nfs-002 1/1 Running 0 19s
2、进入容器创建测试数据
root@k8s-master01:~/learning-k8s/examples/volumes# kubectl exec -it redis-nfs-002 /bin/sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/data # redis-c
redis-check-aof redis-check-rdb redis-cli
/data # redis-cli
127.0.0.1:6379> SET mykey test
OK
127.0.0.1:6379> BGSAVE
Background saving started
127.0.0.1:6379> EXIT
/data # ls
dump.rdb test
3、删除Pod，验证数据持久性
- 第一次nfs Pod被调度在k8s-node02 节点上。
root@k8s-master01:~/learning-k8s/examples/volumes# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
redis-nfs-002 1/1 Running 0 7m49s 10.244.1.4 k8s-node02 <none> <none>
- 删除Pod,并重新创建，人为调度到k8s-node01节点上
root@k8s-master01:~/learning-k8s/examples/volumes# kubectl delete -f pod-with-nfs-vol.yaml
pod "redis-nfs-002" deleted
root@k8s-master01:~/learning-k8s/examples/volumes# cat pod-with-nfs-vol.yaml
apiVersion: v1
kind: Pod
metadata:
name: redis-nfs-002
spec:
nodeName: k8s-node01
containers:
- name: redis
image: redis:7-alpine
imagePullPolicy: IfNotPresent
volumeMounts:
- name: redisdata
mountPath: /data
volumes:
- name: redisdata
nfs:
server: 192.168.10.13
path: /data/redis
root@k8s-master01:~/learning-k8s/examples/volumes# kubectl apply -f pod-with-nfs-vol.yaml
pod/redis-nfs-002 created
root@k8s-master01:~/learning-k8s/examples/volumes# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
redis-nfs-002 1/1 Running 0 4s 10.244.2.31 k8s-node01 <none> <none>
- 进入容器查看数据没有丢失，实现跨界点的数据持久性
root@k8s-master01:~/learning-k8s/examples/volumes# kubectl exec -it redis-nfs-002 /bin/sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/data # ls
dump.rdb test
/data # redis-cli
127.0.0.1:6379> GET mykey
"test"
127.0.0.1:6379>