k8s-系列：1.镜像仓库harbor之ansible-playbook安装

一.准备环境：

1.centos7 环境

2.安装ansible环境

3.harbor安装文件下载路径：

      https://ghproxy.com/https://github.com/goharbor/harbor/releases/download/v2.5.3/harbor-offline-installer-v2.5.3.tgz

4.harbor安装，作者用192.168.126.129作为harbor安装环境

  harbor安装分为两步：

  - a. 安装docker

  - b.安装harbor

二.规划：

1.安装变量规划：

docker安装路径：   docker_dir: "/kingdee/docker"

harbor安装路径：  harbor_dir: "/kingdee/harbor"

harbor域名： harbor_domain： myharbor.pro.com

harbor https端口： harbor_https_port: 10443

harbor http端口： harbor_http_port: 18080

harbor admin用户密码： harbor_password:  Mytest@@@@@1203

harbor目录规划：

安装目录：roles/harbor/{files,tasks,templates,vars}

三.编写 ansible-playbook：

1.创建ansible-playbook剧本目录

mkdir -p roles/harbor/{files,tasks,templates,vars}

2.创建模板文件：

cd roles/templates/

#安装docker-ce源配置文件

vim docker-ce.repo.j2

[docker-ce-stable]
name=Docker CE Stable - $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/stable
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg

[docker-ce-stable-debuginfo]
name=Docker CE Stable - Debuginfo $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/stable
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg

[docker-ce-stable-source]
name=Docker CE Stable - Sources
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/stable
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg

[docker-ce-test]
name=Docker CE Test - $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg

[docker-ce-test-debuginfo]
name=Docker CE Test - Debuginfo $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg

[docker-ce-test-source]
name=Docker CE Test - Sources
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/test
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg

[docker-ce-nightly]
name=Docker CE Nightly - $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/nightly
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg

[docker-ce-nightly-debuginfo]
name=Docker CE Nightly - Debuginfo $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/nightly
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg

[docker-ce-nightly-source]
name=Docker CE Nightly - Sources
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/nightly
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg

#安装epel源配置文件

vim epel.repo.j2

[epel]
name=Extra Packages for Enterprise Linux 7 - $basearch
# It is much more secure to use the metalink, but if you wish to use a local mirror
# place its address here.
#baseurl=http://download.example/pub/epel/7/$basearch
metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch&infra=$infra&content=$contentdir
failovermethod=priority
enabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7

[epel-debuginfo]
name=Extra Packages for Enterprise Linux 7 - $basearch - Debug
# It is much more secure to use the metalink, but if you wish to use a local mirror
# place its address here.
#baseurl=http://download.example/pub/epel/7/$basearch/debug
metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-7&arch=$basearch&infra=$infra&content=$contentdir
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
gpgcheck=0

[epel-source]
name=Extra Packages for Enterprise Linux 7 - $basearch - Source
# It is much more secure to use the metalink, but if you wish to use a local mirror
# place it's address here.
#baseurl=http://download.example/pub/epel/7/source/tree/
metalink=https://mirrors.fedoraproject.org/metalink?repo=epel-source-7&arch=$basearch&infra=$infra&content=$contentdir
failovermethod=priority
enabled=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7
gpgcheck=0

#docker配置文件

vim docker.service.j2

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target docker.socket firewalld.service containerd.service time-set.target
Wants=network-online.target containerd.service
Requires=docker.socket

[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd:// --insecure-registry {{harbor_domain}}:{{harbor_https_port}} --data-root={{docker_dir}} --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutStartSec=0
RestartSec=2
Restart=always

# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3

# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s

# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
TasksMax=infinity

# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes

# kill only the docker process, not all processes in the cgroup
KillMode=process
OOMScoreAdjust=-500

[Install]
WantedBy=multi-user.target

#harbor配置文件

vim harbor.yml.j2

# Configuration file of Harbor
hostname: {{harbor_domian}}
http:
  port: {{harbor_http_port}}
https:
  port: {{harbor_https_port}}
  certificate: {{harbor_dir}}/cert/{{harbor_domain}}.crt
  private_key: {{harbor_dir}}/cert/{{harbor_domain}}.key
harbor_admin_password: {{harbor_password}}
# Harbor DB configuration
database:
  password: {{harbor_password}}
  max_idle_conns: 100
  max_open_conns: 900
# The default data volume
data_volume: {{harbor_dir}}/data
trivy:
  ignore_unfixed: false
  skip_update: false
  offline_scan: false
  insecure: false
jobservice:
  max_job_workers: 10
notification:
  webhook_job_max_retry: 10
chart:
  absolute_url: disabled
# Log configurations
log:
  level: info
  local:
    rotate_count: 50
    rotate_size: 200M
    location: {{harbor_dir}}/logs
#This attribute is for migrator to detect the version of the .cfg file, DO NOT MODIFY!
_version: 2.5.0
proxy:
  http_proxy:
  https_proxy:
  no_proxy:
  components:
    - core
    - jobservice
    - trivy
# enable purge _upload directories
upload_purging:
  enabled: true
  age: 168h
  interval: 24h
  dryrun: false

四.拿取harbor安装文件

cd roles/harbor/files

wget wget https://ghproxy.com/https://github.com/goharbor/harbor/releases/download/v2.5.3/harbor-offline-installer-v2.5.3.tgz -O harbor-offline-installer-v2.5.3.tgz

#生产cer == key: 注意myharbor.pro.com为harbor域名；建议保留改证书，后期安装其他docker需要
openssl genrsa -out myharbor.pro.com.key 4096

openssl req -x509 -new -nodes -sha512 -days 36500 \
 -subj "/C=CN/ST=Guangdongsheng/L=Shenzheng/O=example/OU=Personal/CN=myharbor.pro.com" \
 -key myharbor.pro.com.key \
 -out myharbor.pro.com.crt

五. 编辑任务文件

cd roles/docker/tasks

#docker 安装任务

vim install_docker.yml

---
- name: "copy docker.repo to {{ansible_ssh_host}}"
  template:
    src:  "{{item}}"
    dest: "/etc/yum.repos.d/{{item.split('.j2')[0]}}"
  with_items:
    - docker-ce.repo.j2
    - epel.repo.j2
  become: yes

- name: "yum clean all"
  shell: yum clean all && yum makecache
  become: yes
  
- name: "config sysctl"
  shell: |
    echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf
    sysctl -p
  become: yes

- name: "yum install docker-ce"
  yum:
    name: ['docker-ce','docker-compose']
    state: latest
    enablerepo: docker-ce-stable
  become: yes

- name: "copy docker.config"
  template:
    src: docker.service.j2
    dest: /usr/lib/systemd/system/docker.service
  become: yes

- name: "systemctl daemon-reload"
  shell: systemctl daemon-reload
  become: yes

- name: "mkdir {{docker_dir}}"
  file:
    path: "docker_dir"
    state: directory
  become: yes

- name: "start docker"
  service:
    name: docker
    state: reloaded
    enabled: yes
  become: yes

#harbor安装文件

vim install_harbor.yml

---
- name: "mkdir -p {{harbor_dir}}"
  file:
    path: "{{harbor_dir}}/{{item}}"
    state: directory
    recurse: true
  with_items:
    - cert
    - install_packge
  become: yes

- name: "copy harbor file to {{ansible_ssh_host}}"
  copy:
    src: harbor-offline-installer-v2.5.3.tgz
    dest: /tmp/harbor-offline-installer-v2.5.3.tgz
  become: yes

- name: "unzip harbor-offline-installer-v2.5.3.tgz"
  shell: |
    tar -zxvf /tmp/harbor-offline-installer-v2.5.3.tgz -C {{harbor_dir}}/install_packge
  become: yes

- name: "copy harbor.yml"
  template:
    src: harbor.yml.j2
    dest: "{{harbor_dir}}/install_packge/harbor/harbor.yml"
  become: yes

- name: "copy self-signed cert"
  copy:
    src: "{{item}}"
    dest: "{{harbor_dir}}/cert/{{item}}"
  with_items:
    - "{{harbor_domian}}.crt"
    - "{{harbor_domian}}.key"
  become: yes

- name: "mkdir /etc/docker/{{harbor_domian}} /etc/containerd/{{harbor_domian}}"
  file:
    path: "{{item}}"
    state: directory
    recurse: true
  with_items:
    - "/etc/docker/{{harbor_domian}}"
    - "/etc/containerd/{{harbor_domian}}"
  become: yes

- name: "copy cert to /etc/docker"
  copy:
    src: "{{item}}"
    dest: "/etc/docker/{{harbor_domian}}/{{item}}"
  with_items:
    - "{{harbor_domian}}.crt"
    - "{{harbor_domian}}.key"
  become: yes

- name: "copy cert to /etc/containerd"
  copy:
    src: "{{item}}"
    dest: "/etc/containerd/{{harbor_domian}}/{{item}}"
  with_items:
    - "{{harbor_domian}}.crt"
    - "{{harbor_domian}}.key"
  become: yes

- name: "restart docker"
  service:
    name: docker
    state: restarted
  become: yes

- name: "install harbor"
  shell: |
    cd {{harbor_dir}}/install_packge/harbor && sh install.sh
  become: yes

- name: "add hosts to /etc/hosts"
  shell: |
    echo "{{ansible_ssh_host}} {{harbor_domian}}" >> /etc/hosts
  become: yes


- name: "docker login harbor"
  shell: |
    docker login -u admin -p {{harbor_password}} {{harbor_domian}}:{{harbor_https_port}}
  become: yes



- name: "print docker login info"
  debug:
    msg: "docker login -u admin -p {{harbor_password}} {{harbor_domian}}:'{{harbor_https_port}}'"

#编写主任务文件

vim main.yml

---
- include_tasks: install_docker.yml
- include_tasks: install_harbor.yml

整体目录如下：

六.编写安装运行文件，和roles处于同级目录

vim startinstall_harbor

---
- hosts: harbor
  become: no
  gather_facts: no
  remote_user: yzj
  vars:
    - docker_dir: "/kingdee/docker"
    - harbor_https_port: "10443"
    - harbor_http_port: "18080"
    - harbor_domian: "myharbor.pro.com"
    - harbor_dir: "/kingdee/harbor"
    - harbor_password: "Mytest@@@@@1203"

  roles:
    - role: harbor

vim hosts

[harbor]
k8s-harbor ansible_ssh_host=192.168.126.129

七.执行剧本，安装docker、harbor

ansible-playbook -i hosts startinstall_harbor.yml

安装docker部分：

安装harbor部分：

浏览器访问：

本节介绍完毕，下面会正式进入到k8s-master,k8s-node节点的ansible-playbook部署！