k8s集群-sealos快速部署

下载sealos工具

官方文档页面
https://sealos.run/docs/self-hosting/lifecycle-management/quick-start/install-cli
wget https://github.com/labring/sealos/releases/download/v4.3.7/sealos_4.3.7_linux_amd64.tar.gz

部署

1、配置好ssh免密后进行部署
2、国内机器需要添加如下hosts记录，需要有代理节点上网

172.16.0.12 github.com githubusercontent.com objects.githubusercontent.com registry.cn-shanghai.aliyuncs.com dockerauth.cn-hangzhou.aliyuncs.com aliregistry-cn-shanghai.oss-cn-shanghai.aliyuncs.com

3、安装命令默认没有指定pod和svc的cidr，当前指定了k8s版本 helm版本 cilium版本

sealos run registry.cn-shanghai.aliyuncs.com/labring/kubernetes:v1.27.13 registry.cn-shanghai.aliyuncs.com/labring/helm:v3.9.4 registry.cn-shanghai.aliyuncs.com/labring/cilium:v1.13.4      --masters 172.27.64.100   --nodes 172.27.65.10,172.27.65.11,172.27.65.12 -i .ssh/id_rsa

4、修改node_exporter默认端口

sed -i 's/9100/9101/g' /etc/systemd/system/node-exporter.service
systemctl daemon-reload
systemctl restart node-exporter
sleep 2
curl 127.0.0.1:9101/metrics -I

5、修改containerd数据目录

mkdir /data/ -p
systemctl stop containerd.service 
systemctl stop kubelet

sed -i 's/var\/lib/data/g'  /etc/containerd/config.toml 
mv /var/lib/containerd /data/containerd
systemctl start containerd.service 
systemctl start kubelet

6、helm升级
默认安装的helm版本和1.27.13 k8s兼容性不好,升级到了3.12.3，安装新版本以后替换一下二进制进行。
https://helm.sh/zh/docs/topics/version_skew/

wget https://mirrors.huaweicloud.com/helm/v3.12.3/helm-v3.12.3-linux-amd64.tar.gz

echo 'source <(helm completion bash)' >>~/.bashrc

echo 'source <(kubectl completion bash)' >>~/.bashrc

7、nfs安装

# nfs服务器安装
apt-get install nfs-kernel-server nfs-common

# 写入配置
echo "/data/kube-nfs *(insecure,rw,async,no_root_squash)" | sudo tee -a /etc/exports

# 重新加载 NFS 服务器配置
sudo exportfs -ra

# 启动 NFS 服务器
sudo systemctl start nfs-kernel-server

# 设置 NFS 服务器开机启动
sudo systemctl enable nfs-kernel-server
# nfs客户端安装
apt-get install nfs-common

由于网络问题导致镜像无法正常拉取，只能导入镜像后使用helm指定多个参数安装

# helm部署（国内网络镜像下不来）
helm repo add nfs-subdir-external https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
helm repo update
helm install nfs-client nfs-subdir-external/nfs-subdir-external-provisioner --set nfs.server=172.27.65.10 --set nfs.path=/data/kube-nfs


镜像pull error，手动加hosts以后node-12下载成功了，然后指定镜像名称，这样可能优先使用本地image了
helm install nfs-client nfs-subdir-external/nfs-subdir-external-provisioner --set nfs.server=172.27.65.10 --set nfs.path=/data/kube-nfs --set image.repository=registry.k8s.io/sig-storage/nfs-subdir-external-provisioner --set image.tag=v4.0.2

测试pvc

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
  annotations:
    volume.beta.kubernetes.io/storage-class: "nfs-client"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Mi

8、ingress安装
测试ingress使用默认http协议，并且configmap中需要开启svc的访问
生产ingress使用tcp proxy协议，分出2-3种ingress，内部、外部-静态、外部api
ingress选择k8s兼容的版本https://github.com/kubernetes/ingress-nginx