快速搭建 K8s 集群
| 角色 | ip |
|---|---|
| k8s-master-01 | 192.168.111.170 |
| k8s-node-01 | 192.168.111.171 |
| k8s-node-02 | 192.168.111.172 |
服务器需要连接互联网下载镜像
| 软件 | 版本 |
|---|---|
| Docker | 24.0.0（CE） |
| Kubernetes | 1.28 |
初始化配置
关闭防火墙
systemctl stop firewalld && systemctl disable firewalld
关闭Selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0
关闭Swap
sed -ri 's/.*swap.*/#&/' /etc/fstab
swapoff -a
根据规划设置主机名
hostnamectl set-hostname k8s-master-01
hostnamectl set-hostname k8s-node-01
hostnamectl set-hostname k8s-node-02
网络桥段
vi /etc/sysctl.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-arptables = 1
net.ipv4.ip_forward=1
net.ipv4.ip_forward_use_pmtu = 0
# 生效命令
sysctl --system
# 查看效果
sysctl -a|grep "ip_forward"
确保网络桥接的数据包经过Iptables处理，防止网络丢包
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system # 生效
同步时间
# 安装软件
yum -y install ntpdate
# 向阿里云服务器同步时间
ntpdate time1.aliyun.com
# 删除本地时间并设置时区为上海
rm -rf /etc/localtime && ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
# 查看时间
date -R || date
开启 IPVS
yum -y install ipset ipvsdm
cat > /etc/sysconfig/modules/ipvs.modules << EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF
# 赋予权限并执行
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules &&lsmod | grep -e ip_vs -e nf_conntrack_ipv4
# 重启电脑，检查是否生效
reboot
命令补全
yum -y install bash-completion bash-completion-extras
source /etc/profile.d/bash_completion.sh
配置 HOSTS
cat <<EOF >>/etc/hosts
192.168.111.170 k8s-master-01
192.168.111.171 k8s-node-01
192.168.111.172 k8s-node-02
EOF
cat <<EOF >>/etc/hosts
192.168.111.173 k8s-tool-01 harbor.liuyuncen.com
EOF
安装docker
下载源
yum install -y wget
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
安装 docker
# yum list docker-ce --showduplicates | sort -r
yum -y install docker-ce-24.0.0
systemctl enable docker && systemctl start docker
设置Cgroup驱动
cat > /etc/docker/daemon.json << EOF
{
"exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
systemctl daemon-reload && systemctl restart docker
# 查看设置状态
docker info
安装 cri-docker 驱动 （Docker与Kubernetes通信的中间程序）：
# wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.2/cri-dockerd-0.3.2-3.el7.x86_64.rpm
rpm -ivh cri-dockerd-0.3.2-3.el7.x86_64.rpm
指定依赖镜像地址为国内镜像地址：
vim /usr/lib/systemd/system/cri-docker.service
ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.9
systemctl daemon-reload
systemctl enable cri-docker && systemctl start cri-docker
部署 K8s 集群
添加阿里云 yum 源
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
安装kubeadm 官方提供的集群搭建工具，kubelet 守护进程 和kubectl 管理集群工具（实际在 master 安装即可）
yum install -y kubelet-1.28.0 kubeadm-1.28.0 kubectl-1.28.0
systemctl enable kubelet
# 这里只是设置开机启动，直接起也起不来
提前把所有镜像都拉下来
docker load -i calico.v3.25.1.tar
只需要 k8s-master-01 节点执行 初始化Master节点
kubeadm init \
--apiserver-advertise-address=192.168.126.170 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.28.0 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=10.244.0.0/16 \
--cri-socket=unix:///var/run/cri-dockerd.sock
初始化信息
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
只需要 k8s-node-01 、k8s-node-02 节点执行
--cri-socket=unix:///var/run/cri-dockerd.sock 这一句要补充在最后面
kubeadm join 192.168.111.170:6443 --token y8hujn.777t21thlk6v6hy0 --discovery-token-ca-cert-hash sha256:f8df8dfe6cb7ad5347f92b6c58f552df8982c7dce540b266c22f971e49f55684 --cri-socket=unix:///var/run/cri-dockerd.sockkubeadm join 192.168.126.170:6443 --token puuo90.pqgbmu1d32x3vsq4 --discovery-token-ca-cert-hash sha256:3600ff21c5a3742fee0455d920f345ea0ab9c99f0356acf7675f2ee728448bff --cri-socket=unix:///var/run/cri-dockerd.sock
使用kubectl工具查看节点状态： kubectl get nodes 由于网络插件还没有部署，节点会处于“NotReady”状态
这里使用Calico作为Kubernetes的网络插件，负责集群中网络通信。创建Calico网络组件的资源
kubectl create -f tigera-operator.yaml
kubectl create -f custom-resources.yaml
执行完成后，等待几分钟，执行
kubectl get pods -n calico-system -o wide
问题排查，发现 nodes 无法启动，原因是下载不了镜像
kubectl describe po calico-kube-controllers-85955d4f5b-dbhhr -n calico-system kubectl describe po calico-node-2sdxr -n calico-system kubectl describe po calico-node-65gw4 -n calico-system kubectl describe po calico-node-xqvnf -n calico-system kubectl describe po calico-typha-7cd7bb8d58-lqmxj -n calico-system kubectl describe po calico-typha-7cd7bb8d58-zwjd4 -n calico-system kubectl describe po csi-node-driver-d9vkx -n calico-system kubectl describe po csi-node-driver-nl26v -n calico-system kubectl describe po csi-node-driver-qzljb -n calico-systemsystemctl daemon-reload && systemctl restart dockerdocker pull registry.cn-beijing.aliyuncs.com/yuncenliu/cni:v3.25.1-calico docker tag registry.cn-beijing.aliyuncs.com/yuncenliu/cni:v3.25.1-calico docker.io/calico/pod2daemon-flexvol:v3.25.1 docker pull registry.cn-beijing.aliyuncs.com/yuncenliu/pod2daemon-flexvol:v3.25.1-calico docker tag registry.cn-beijing.aliyuncs.com/yuncenliu/pod2daemon-flexvol:v3.25.1-calico docker.io/calico/cni:v3.25.1重启containerd （有用）
systemctl restart containerd删除失败的 pods
kubectl get pods -n kube-system | grep calico-node-bvvhc | awk '{print$1}'| xargs kubectl delete -n kube-system podsdocker save -o calico.v3.25.1.tar calico/kube-controllers:v3.25.1 calico/node:v3.25.1 calico/pod2daemon-flexvol:v3.25.1 calico/cni:v3.25.1 docker load -i calico.v3.25.1.tar
全部启动成功，节点也全都在线 kubectl get nodes
安装页面
所有节点先下拉镜像
docker load -i k8s-dashboard-v2.7.0.tar
创建 Dashboard ，在 master 节点执行
kubectl apply -f kubernetes-dashboard.yaml
kubectl get pods -n kubernetes-dashboard -o wide
执行完成后，任意 https://任意node:30001 访问，例如：https://192.168.111.170:30001/#/login
创建用户
kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard
用户授权
kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:dashboard-admin
获取用户 token
kubectl create token dashboard-admin -n kubernetes-dashboard
将 token 粘贴到浏览器的 token输入栏中（默认还是选第一项）点击登录即可
