首页 > 其他分享 >k8s集群安装nfs持久化存储

k8s集群安装nfs持久化存储

时间：2024-03-07 12:01:05浏览次数：30  
标签：k8s server nginx 集群 master nfs kube root

k8s集群安装nfs-server服务

  • 下载并修改yaml
mkdir -p /root/yaml/addons/nfs
cd /root/yaml/addons/nfs
wget https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/deploy/example/nfs-provisioner/nfs-server.yaml
  • 修改yaml内容，添加namespace
[root@kube-master nfs]# cat nfs-server.yaml 
---
apiVersion: v1
kind: Namespace
metadata:
  name: nfs
---
kind: Service
apiVersion: v1
metadata:
  name: nfs-server
  namespace: nfs
  labels:
    app: nfs-server
spec:
  type: ClusterIP  # use "LoadBalancer" to get a public ip
  selector:
    app: nfs-server
  ports:
    - name: tcp-2049
      port: 2049
      protocol: TCP
    - name: udp-111
      port: 111
      protocol: UDP
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs-server
  namespace: nfs
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-server
  template:
    metadata:
      name: nfs-server
      labels:
        app: nfs-server
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
        - name: nfs-server
          image: itsthenetwork/nfs-server-alpine:latest
          env:
            - name: SHARED_DIRECTORY
              value: "/exports"
          volumeMounts:
            - mountPath: /exports
              name: nfs-vol
          securityContext:
            privileged: true
          ports:
            - name: tcp-2049
              containerPort: 2049
              protocol: TCP
            - name: udp-111
              containerPort: 111
              protocol: UDP
      volumes:
        - name: nfs-vol
          hostPath:
            path: /nfs-vol  # modify this to specify another path to store nfs share data
            type: DirectoryOrCreate
[root@kube-master nfs]#
  • 部署nfs-server
[root@kube-master nfs]# kubectl apply -f nfs-server.yaml 
namespace/nfs created
service/nfs-server created
deployment.apps/nfs-server created
  • 检查nfs-server资源状态
[root@kube-master nfs]# ls -l /nfs-vol/
total 0
[root@kube-master nfs]# kubectl get all -n nfs
NAME                              READY   STATUS    RESTARTS   AGE
pod/nfs-server-7cc5bcdcd5-rnwgr   1/1     Running   0          56s

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)            AGE
service/nfs-server   ClusterIP   10.97.192.219   <none>        2049/TCP,111/UDP   56s

NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nfs-server   1/1     1            1           56s

NAME                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/nfs-server-7cc5bcdcd5   1         1         1       56s

k8s集群安装csi-driver-nfs

  • 下载
git clone https://github.com/kubernetes-csi/csi-driver-nfs.git
cd csi-driver-nfs/
  • 修改镜像仓库为阿里云
[root@kube-master csi-driver-nfs]# ls -ld deploy/v*
drwxr-xr-x 2 root root 129 Mar  7 09:35 deploy/v3.0.0
drwxr-xr-x 2 root root 129 Mar  7 09:35 deploy/v3.1.0
drwxr-xr-x 2 root root 129 Mar  7 09:35 deploy/v4.0.0
drwxr-xr-x 2 root root 118 Mar  7 09:35 deploy/v4.1.0
drwxr-xr-x 2 root root 118 Mar  7 09:35 deploy/v4.2.0
drwxr-xr-x 2 root root 220 Mar  7 09:35 deploy/v4.3.0
drwxr-xr-x 2 root root 220 Mar  7 09:35 deploy/v4.4.0
drwxr-xr-x 2 root root 220 Mar  7 09:35 deploy/v4.5.0
drwxr-xr-x 2 root root 220 Mar  7 09:35 deploy/v4.6.0
[root@kube-master csi-driver-nfs]# cp -a deploy/v4.6.0/ deploy/v4.6.0-bak
sed -i '[email protected]/[email protected]/google_containers@g' deploy/v4.6.0/*.yaml
  • 安装nfs csi driver
[root@kube-master csi-driver-nfs]# ./deploy/install-driver.sh v4.6.0 local
use local deploy
Installing NFS CSI driver, version: master ...
serviceaccount/csi-nfs-controller-sa created
serviceaccount/csi-nfs-node-sa created
clusterrole.rbac.authorization.k8s.io/nfs-external-provisioner-role created
clusterrolebinding.rbac.authorization.k8s.io/nfs-csi-provisioner-binding created
csidriver.storage.k8s.io/nfs.csi.k8s.io created
deployment.apps/csi-nfs-controller created
daemonset.apps/csi-nfs-node created
NFS CSI driver installed successfully.
[root@kube-master csi-driver-nfs]#
  • 查看nfs csi资源的状态
[root@kube-master csi-driver-nfs]# kubectl -n kube-system get pod -o wide |grep csi-nfs
csi-nfs-controller-6f5f88ff49-8rsrz   4/4     Running   0              4m41s   192.168.76.135   kube-master   <none>           <none>
csi-nfs-node-xmjgh                    3/3     Running   0              4m41s   192.168.76.135   kube-master   <none>           <none>
  • 【可选】卸载nfs csi
./deploy/uninstall-driver.sh v4.6.0 local

快速测试nfs-server及nfs csi可用性

  • test-nfs.yaml
---
apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    pv.kubernetes.io/provisioned-by: nfs.csi.k8s.io
  name: pv-nginx
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Delete
  mountOptions:
    - nfsvers=4.1
  csi:
    driver: nfs.csi.k8s.io
    # volumeHandle format: {nfs-server-address}#{sub-dir-name}#{share-name}
    # make sure this value is unique for every share in the cluster
    volumeHandle: nfs-server.nfs.svc.cluster.local/share##
    volumeAttributes:
      server: nfs-server.nfs.svc.cluster.local
      share: /nfs-vol
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-nginx
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  volumeName: pv-nginx
  storageClassName: ""
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx-nfs-example
spec:
  containers:
    - image: nginx
      name: nginx
      ports:
        - containerPort: 80
          protocol: TCP
      volumeMounts:
        - mountPath: /var/www
          name: pvc-nginx
          readOnly: false
  volumes:
    - name: pvc-nginx
      persistentVolumeClaim:
        claimName: pvc-nginx
  • 创建pv,pvc并测试pod
[root@kube-master nfs]# kubectl apply -f test-nfs.yaml 
persistentvolume/pv-nginx created
persistentvolumeclaim/pvc-nginx created
pod/nginx-nfs-example created
[root@kube-master nfs]# kubectl exec -it nginx-nfs-example -- touch /var/www/a.log
[root@kube-master nfs]# ls -l /nfs-vol/
total 0
-rw-r--r-- 1 root root 0 Mar  7 11:09 a.log
[root@kube-master nfs]# rm -f /nfs-vol/a.log 
[root@kube-master nfs]# kubectl exec -it nginx-nfs-example -- ls -l /var/www/
total 0
[root@kube-master nfs]#
  • 查看nfs挂载信息
[root@kube-master nfs]# kubectl exec nginx-nfs-example -- bash -c "findmnt /var/www -o TARGET,SOURCE,FSTYPE"
TARGET   SOURCE                             FSTYPE
/var/www nfs-server.nfs.svc.cluster.local:/ nfs4
  • 删除测试资源
[root@kube-master nfs]# kubectl delete -f test-nfs.yaml 
persistentvolume "pv-nginx" deleted
persistentvolumeclaim "pvc-nginx" deleted
pod "nginx-nfs-example" deleted

测试storage class

  • 创建storageclass-nfs.yaml
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-sc
provisioner: nfs.csi.k8s.io
parameters:
  server: nfs-server.nfs.svc.cluster.local
  share: /
reclaimPolicy: Retain
volumeBindingMode: Immediate
mountOptions:
  - nfsvers=4.1
  • 查看sc状态
[root@kube-master nfs]# kubectl apply -f nfs-sc.yaml 
storageclass.storage.k8s.io/nfs-sc created
[root@kube-master nfs]# kubectl get sc
NAME     PROVISIONER      RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-sc   nfs.csi.k8s.io   Retain          Immediate           false                  10s
  • 创建pvc
[root@kube-master nfs]# cat pvc-nginx-log.yaml 
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-nginx-log
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: nfs-sc
[root@kube-master nfs]# kubectl apply -f pvc-nginx-log.yaml 
persistentvolumeclaim/pvc-nginx-log created
  • 创建test-nfs-sc.yaml
[root@kube-master nfs]# cat test-nfs-sc.yaml 
---
apiVersion: v1
kind: Pod
metadata:
  name: nginx-nfs-example
spec:
  containers:
    - image: nginx
      name: nginx
      ports:
        - containerPort: 80
          protocol: TCP
      volumeMounts:
        - mountPath: /var/log/nginx/
          name: nginx-log
          readOnly: false
  volumes:
    - name: nginx-log
      persistentVolumeClaim:
        claimName: pvc-nginx-log
[root@kube-master nfs]# kubectl apply -f test-nfs-sc.yaml
pod/nginx-nfs-example created
[root@kube-master nfs]# 
[root@kube-master nfs]# kubectl get pvc
NAME            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
pvc-nginx-log   Bound    pvc-cead6357-bea8-421d-a31b-78a1c715a427   1Gi        RWX            nfs-sc         <unset>                 4m24s
[root@kube-master nfs]# kubectl get pods -o wide
NAME                READY   STATUS    RESTARTS       AGE   IP            NODE          NOMINATED NODE   READINESS GATES
nginx-nfs-example   1/1     Running   0              19s   10.244.0.30   kube-master   <none>           <none>
  • 查看nginx日志是否保存到nfs-server
[root@kube-master nfs]# curl 10.244.0.30
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@kube-master nfs]# ls -l /nfs-vol/
total 0
drwxr-xr-x 2 root root 41 Mar  7 11:44 pvc-cead6357-bea8-421d-a31b-78a1c715a427
[root@kube-master nfs]# ls -l /nfs-vol/pvc-cead6357-bea8-421d-a31b-78a1c715a427/
total 4
-rw-r--r-- 1 root root   0 Mar  7 11:44 access.log
-rw-r--r-- 1 root root 515 Mar  7 11:45 error.log
  • 删除pod，并新建pod，看nfs-server文件夹的状态
[root@kube-master nfs]# kubectl delete -f test-nfs-sc.yaml 
persistentvolumeclaim "pvc-nginx-log" deleted
pod "nginx-nfs-example" deleted
[root@kube-master nfs]# ls -l /nfs-vol/
total 0
drwxr-xr-x 2 root root 41 Mar  7 11:27 pvc-cfa95fab-9cf0-4acd-8699-1a123d218741
[root@kube-master nfs]# ls -l /nfs-vol/pvc-cfa95fab-9cf0-4acd-8699-1a123d218741/
total 8
-rw-r--r-- 1 root root  273 Mar  7 11:31 access.log
-rw-r--r-- 1 root root 1268 Mar  7 11:32 error.log
[root@kube-master nfs]# 
[root@kube-master nfs]# kubectl apply -f test-nfs-sc.yaml 
pod/nginx-nfs-example created
[root@kube-master nfs]# kubectl get pods -o wide
NAME                READY   STATUS    RESTARTS        AGE    IP            NODE          NOMINATED NODE   READINESS GATES
myapp1              2/2     Running   4 (3h10m ago)   6d1h   10.244.0.21   kube-master   <none>           <none>
nginx-nfs-example   1/1     Running   0               117s   10.244.0.31   kube-master   <none>           <none>
[root@kube-master nfs]# 
[root@kube-master nfs]# 
[root@kube-master nfs]# curl 10.244.0.31
[root@kube-master nfs]# cat /nfs-vol/pvc-cead6357-bea8-421d-a31b-78a1c715a427/access.log 
10.244.0.1 - - [07/Mar/2024:03:45:22 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.29.0" "-"
10.244.0.1 - - [07/Mar/2024:03:50:00 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.29.0" "-"
10.244.0.1 - - [07/Mar/2024:03:50:03 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.29.0" "-"
10.244.0.1 - - [07/Mar/2024:03:50:03 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.29.0" "-"
10.244.0.1 - - [07/Mar/2024:03:50:04 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.29.0" "-"
10.244.0.1 - - [07/Mar/2024:03:50:04 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.29.0" "-"
10.244.0.1 - - [07/Mar/2024:03:50:05 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.29.0" "-"
10.244.0.1 - - [07/Mar/2024:03:50:06 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.29.0" "-"

参考文档

标签：k8s,server,nginx,集群,master,nfs,kube,root
From： https://www.cnblogs.com/hello-littlebaby/p/18058557

相关文章

  • Elasticsearch 集群网络配置实例
    网络配置在elasticsearch中，客户端通过http与es进行通信；es集群节点间主要通过transport进行通信。在不同的网络环境下，需要进行相应的网络配置调整。简单网络环境单网卡假设只有一个网络接口A，IP地址为176.33.2.101、http通信端口9200、transport通信端口9300。#监听的IPnetw......
  • K8S-Pod控制器
    Pod控制器Pod控制器及其功用Pod控制器，又称之为工作负载（workload），是用于实现管理pod的中间层，确保pod资源符合预期的状态，pod的资源出现故障时，会尝试进行重启，当根据重启策略无效，则会重新新建pod的资源。pod控制器有多种类型1、ReplicaSet:代用户创建指定数量的pod副本，确保pod......
  • Elasticsearch集群生产配置
    集群配置在组建集群时，需要额外添加集群相关的配置，如节点角色、集群发现、初始主节点、主节点选举和安全认证等，以下配置均在上篇创建Elasticsearch单机实例的基础上搭建。设置集群名称vielasticsearch.yml#统一集群名字cluster.name:my-application#移除该配置discover......
  • kubenetes系列-部署csi-driver-nfs
    1、部署helm#这里要注意因为下周的是二进制直接可用，需要根据系统去下载wgethttps://get.helm.sh/helm-v3.14.2-linux-amd64.tar.gztarxfhelm-v3.14.2-linux-amd64.tar.gz&&cdlinux-arm64&&mvhelm/usr/bin/2、使用helm部署helmpullcsi-driver-nfs/csi-driver-nfs......
  • 极狐GitLab Runner Kubernetes（k8s）配置
    GitLab是一个全球知名的一体化DevOps平台，很多人都通过私有化部署GitLab来进行源代码托管。极狐GitLab是GitLab在中国的发行版，专门为中国程序员服务。可以一键式部署极狐GitLab。资料Kubernetes高级配置添加额外主机别名Pod的DNS配置极狐GitLabRunner......
  • 简单懂点k8s网络（4）K8S对外暴露方案
       ......
  • k8s StorageClass(NFS)
    nfs服务器自行一键安装1:创建ServiceAccount，为nfs-client授权。nfs-client-sa.yaml---apiVersion:v1kind:ServiceAccountmetadata:name:nfs-client-provisioner---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRolemetadata:name:nfs-client-provis......
  • K8S --存储卷
    存储卷容器磁盘上的文件的生命周期是短暂的，这就使得在容器中运行重要应用时会出现一些问题。首先，当容器崩溃时，kubelet会重启它，但是容器中的文件将丢失——容器以干净的状态（镜像最初的状态）重新启动。其次，在Pod中同时运行多个容器时，这些容器之间通常需要共享文件。Kubernetes中......
  • K8S部署MySQL5.7主从集群
    参考https://blog.csdn.net/qq_43114229/article/details/124078914https://kubernetes.io/zh-cn/docs/tasks/run-application/run-replicated-stateful-application/MySQL主从同步架构图mysql-0是mastermysql-1和mysql-2是两个备份当mysql写的时候，找headlessservice中的m......
  • K8s中Role(ClusterRole)资源类型rules字段详解
    在Kubernetes（K8s）中，Role资源类型的rules字段用于定义哪些操作（verbs）可以在哪些资源（resources）上执行。Role是一种命名空间级别的资源，它允许你对命名空间内的资源进行细粒度的访问控制。resources：resources字段指定了角色可以访问的资源类型。这些资源类型可以是KubernetesAPI中......

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99