一、概要
提到K8S环境下的CI/CD,可以使用的工具有很多,比如Jenkins、Gitlab CI、新兴的drone等,考虑到大多公司在VM环境下都采用 Jenkins 集群来搭建符合需求的 CI/CD 流程,这里先给介绍大家下Kubernetes+Jenkins的CI/CD方案。
Jenkins是一款开源 CI&CD 系统,用于自动化各种任务,包括构建、测试和部署。
Jenkins官方提供了镜像:https://hub.docker.com/r/jenkins/jenkins
1、Jenkins架构
Jenkins Master 和 Jenkins Slave 以 Pod 形式运行在 Kubernetes 集群的 Node 上,Master是常驻服务,所有的配置数据都存储在一个 Volume 中,Slave 不是一直处于运行状态,它会按照需求动态的创建并自动删除。
2、工作原理
当 Jenkins Master 接受到 Build 请求时,会根据配置的 Label 动态创建一个运行在 Pod 中的 Jenkins Slave 并注册到 Master 上,当运行完 Job 后,这个 Slave 会被注销并且这个 Pod 也会自动删除,恢复到最初状态。
3、优势
相对于部署在虚拟机环境下的Jenkins 一主多从架构,将Jenkins部署到K8S会带来以下好处:
-
服务高可用: 当 Jenkins Master 出现故障时,Kubernetes 会自动创建一个新的 Jenkins Master 容器,并且将 Volume 分配给新创建的容器,保证数据不丢失,从而达到集群服务高可用。
-
动态伸缩: 合理使用资源,每次运行 Job 时,会自动创建一个 Jenkins Slave,Job 完成后,Slave 自动注销并删除容器,资源自动释放,而且 Kubernetes 会根据每个资源的使用情况,动态分配 Slave 到空闲的节点上创建,降低出现因某节点资源利用率高,还排队等待在该节点的情况。
-
扩展性好: 当 Kubernetes 集群的资源严重不足而导致 Job 排队等待时,可以很容易的添加一个 Kubernetes Node 到集群中,从而实现扩展。
二、部署Jenkins
使用Deployment来部署这个镜像,会暴露两个端口:8080 Web访问端口,50000 Slave通信端口,容器启动后Jenkins数据存储在/var/jenkins_home目录,所以需要将该目录使用PV持久化存储。
1、配置静态PV持久化存储:
1.1 部署NFS共享服务器
在所有节点安装NFS软件包:yum install nfs-utils -y
1.2 找一个节点作为NFS共享存储服务器
mkdir -p /ifs/kubernetes/jenkins-data
vi /etc/exports
/ifs/kubernetes 192.168.200.0/24(rw,no_root_squash)
systemctl start nfs && systemctl enable nfs
1.3 为Jenkins准备PV
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv0001
spec:
capacity:
storage: 5Gi
accessModes: ["ReadWriteOnce"]
nfs:
path: /ifs/kubernetes/jenkins-data
server: 192.168.200.31
1.4 编写相应yaml
deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins
namespace: ops
labels:
name: jenkins
spec:
replicas: 1
selector:
matchLabels:
name: jenkins
template:
metadata:
name: jenkins
labels:
name: jenkins
spec:
serviceAccountName: jenkins
containers:
- name: jenkins
image: jenkins/jenkins
ports:
- containerPort: 8080
- containerPort: 50000
resources:
limits:
cpu: 2
memory: 4Gi
requests:
cpu: 1
memory: 1Gi
env:
- name: LIMITS_MEMORY
valueFrom:
resourceFieldRef:
resource: limits.memory
divisor: 1Mi
- name: JAVA_OPTS
value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
volumeMounts:
- name: jenkins-home
mountPath: /var/jenkins_home
securityContext:
fsGroup: 1000
volumes:
- name: jenkins-home
persistentVolumeClaim:
claimName: jenkins
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: jenkins
namespace: ops
spec:
storageClassName: "managed-nfs-storage"
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Gi
rbac.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
namespace: ops
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: jenkins
namespace: ops
rules:
- apiGroups: [""]
resources: ["pods","events"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list","watch"]
- apiGroups: [""]
resources: ["secrets","events"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: jenkins
namespace: ops
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jenkins
subjects:
- kind: ServiceAccount
name: jenkins
service.yml
apiVersion: v1
kind: Service
metadata:
name: jenkins
namespace: ops
spec:
selector:
name: jenkins
type: NodePort
ports:
- name: http
port: 80
targetPort: 8080
protocol: TCP
nodePort: 30008
- name: agent
port: 50000
protocol: TCP
1.4 运行相应yml
kubectl apply -f deployment.yml -f rbac.yml -f service.yml
- Git:拉去代码
- Git Parameter:Git参数化构建
- Pipeline:流水线
- kubernetes:连接Kubernetes动态创建Slave代理
- Config File Provider:存储配置文件
- Extended Choice Parameter:扩展选择框参数,支持多选
2、Jenkins主从架构
2.1 安装相应插件
管理Jenkins --> 系统配置 --> 管理插件 --> 分别搜索 Git/Git Parameter/Pipeline/kubernetes/Config File Provider/Extended Choice Parameter
配置插件: 管理Jenkins --> 管理Node和云 --> 管理云 --> 添加Kubernetes
2.2 自定义Jenkins-Slave镜像
FROM centos:7
LABEL maintainer Xin
RUN yum install -y java-1.8.0-openjdk maven curl git libtool-ltdl-devel && \
yum clean all && \
rm -rf /var/cache/yum/* && \
mkdir -p /usr/share/jenkins
COPY slave.jar /usr/share/jenkins/slave.jar
COPY jenkins-slave /usr/bin/jenkins-slave
COPY settings.xml /etc/maven/settings.xml
RUN chmod +x /usr/bin/jenkins-slave
ENTRYPOINT ["jenkins-slave"]
Dockerfile涉及的六个文件:
- jenkins-slave:shell脚本启动slave.jar,下载地址:https://github.com/jenkinsci/docker-inbound-agent/blob/master/jenkins-slave
- settings.xml:修改maven官方源为阿里云源
- slave.jar:agent程序,接受master下发的任务,下载地址:http://jenkinsip:port/jnlpJars/slave.jar jinkinsip内容换成部署Jenkins服务器的IP地址
- kubectl客户端工具
构建并推送到镜像仓库:
docker build -t 192.168.200.10/library/jenkins-slave-jdk:1.8 .
docker push 192.168.200.10/library/jenkins-slave-jdk:1.8
2.3 测试主从架构是否正常
新建项目 --> 流水线 --> Pipeline脚本
```yaml
pipeline {
agent {
kubernetes {
label "jenkins-slave"
yaml '''
apiVersion: v1
kind: Pod
metadata:
name: jenkins-slave
spec:
containers:
- name: jnlp
image: "192.168.200.10/library/jenkins-slave-jdk:1.8"
'''
}
}
stages {
stage('Main'){
steps {
sh 'hostname'
}
}
}
}
重启Jenkins,在Jenkins域名后+ /_restart
kubectl get pods -n ops
标签:kind,Slave,slave,name,架构,jenkins,Jenkins,主从
From: https://www.cnblogs.com/superingXin/p/17561131.html