基于 VRRP Script 实现其它应用的高可用性
keepalived利用 VRRP Script 技术,可以调用外部的辅助脚本进行资源监控,并根据监控的结果实现优先动态调整,从而实现其它应用的高可用性功能
VRRP Script配置
分两步实现
定义脚本：
vrrp_script：自定义资源监控脚本,vrrp实例根据脚本返回值,公共定义,可被多个实例调用,定 义在vrrp实例之外的独立配置块,一般放在global_defs设置块之后,是和global_defs平级的语句块 通常此脚本用于监控指定应用的状态.一旦发现应用的状态异常,则触发对MASTER节点的权重减 至低于SLAVE节点,从而实现 VIP 切换到 SLAVE 节点调用脚本：
track_script：调用vrrp_script定义的脚本去监控资源,定义在VRRP实例之内,调用事先定义的 vrrp_script定义VRRP Script
vrrp_script <SCRIPT_NAME> { #定义一个检测脚本,在global_defs 之外配置
script <STRING>|<QUOTED-STRING> #shell命令或脚本路径
interval <INTEGER> #间隔时间,单位为秒,默认1秒
timeout <INTEGER> #超时时间
weight <INTEGER:-254..254> #默认为0,如果设置此值为负数,当上面脚本返回值为非0时,会将此值与本节点权重相加可以降低本节点权重,即表示fall. 如果是正数,当脚本返回值为0,会将此值与本节点权重相加可以提高本节点权重,即表示 rise.通常使用负值
fall <INTEGER> #执行脚本连续几次都失败,则转换为失败,建议设为2以上
rise <INTEGER> #执行脚本连续几次都成功,把服务器从失败标记为成功
user USERNAME [GROUPNAME] #执行监测脚本的用户或组
init_fail #设置默认标记为失败状态,监测成功之后再转换 为成功状态 }调用VRRP script
vrrp_instance VI_1 {
…
track_script {
<SCRIPT_NAME>
}
}范例：利用脚本实现主从角色切换（脚本可放主配置、子配置文件）
第一步：把原来lvs的配置文件拷贝改名，保留文件内容，并更改原lvs配置用于其他应用的高可用
[root@ka1 ~]#cd /etc/keepalived/conf.d/
[root@ka1 conf.d]#cp www.meng.org.conf www.meng.org.conf.lvs
[root@ka1 conf.d]#cp www.meng.com.conf www.meng.com.conf.lvs
[root@ka1 conf.d]#ls
www.meng.com.conf www.meng.com.conf.lvs www.meng.org.conf www.meng.org.conf.lvs
第二步：更改ka1的www.meng.org.conf配置文件，启用并引用脚本
[root@ka1 conf.d]#cat www.meng.org.conf
vrrp_script check_down { #check_down,脚本名
script "[ ! -f /etc/keepalived/down ]" #判断文件是否存在，不存在为真，存在为假权重-30
interval 1
weight -30
fall 3
rise 2
timeout 2
}
vrrp_instance VI_1 {
state MASTER
interface eth1
virtual_router_id 66
priority 100
advert_int 1
#nopreempt
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.100/24 dev eth0 label eth0:1
}
unicast_src_ip 192.168.10.100
unicast_peer{
192.168.10.101
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
track_script { #引用脚本
check_down
}
}
[root@ka1 conf.d]#systemctl restart keepalived
第三步：把原来lvs的配置文件拷贝改名，保留文件内容，并更改原lvs配置用于其他应用的高可用
[root@ka2 conf.d]#cp www.meng.org.conf www.meng.org.conf.lvs
[root@ka2 conf.d]#cp www.meng.com.conf www.meng.com.conf.lvs
[root@ka2 conf.d]#ls
www.meng.com.conf www.meng.com.conf.lvs www.meng.org.conf www.meng.org.conf.lvs
第四步：更改ka2的www.meng.org.conf配置文件，启用并引用脚本
[root@ka2 conf.d]#cat www.meng.org.conf
vrrp_script check_down {
script "[ ! -f /etc/keepalived/down ]"
interval 1
weight -30
fall 3
rise 2
timeout 2
}
vrrp_instance VI_1 {
state BACKUP
interface eth1
virtual_router_id 66
priority 80
advert_int 1
#nopreempt
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.150/24 dev eth0 label eth0:1
}
unicast_src_ip 192.168.10.101
unicast_peer{
192.168.10.100
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
track_script { #引用脚本
check_down
}
}
[root@ka2 conf.d]#systemctl restart keepalivedwww.meng.com.conf和上面执行一样的步骤
测试
script "[ ! -f /etc/keepalived/down ]"
当/etc/keepalived/down文件存在则为假，则执行脚本，优先级便会减30
把这个文件创建出来，文件存在，优先级便将为70，则VIP将飘到ka2节点上
[root@ka1 conf.d]#touch /etc/keepalived/down
[root@ka1 conf.d]#hostname -I
10.0.0.101 192.168.10.100
[root@ka2 conf.d]#hostname -I
10.0.0.102 10.0.0.200 10.0.0.100 192.168.10.101实战案例：实现 HAProxy 高可用
第一步：在keepalived机器上安装haproxy
[root@ka1 conf.d]#apt -y install haproxy
[root@ka2 conf.d]#apt -y install haproxy
第二步：修改haproxy的配置文件（增加状态页，设置后端代理服务器地址），把这两项内容加在最后
[root@ka1 conf.d]#vim /etc/haproxy/haproxy.cfg
listen status #增加状态也
bind 10.0.0.100:9999 #访问状态页面默认是10.0.0.100:9999/haproxy?stats
stats enable
listen www.meng.org_80 #设置后端代理服务器地址
bind 10.0.0.100:80
server web01 10.0.0.8:80 check
server web02 10.0.0.18:80 check
语法检查（检查正确还起不来有可能是本机带的nginx端口和设置端口起冲突，或者没有100这个地址，被抢占）
[root@ka1 conf.d]#/usr/sbin/haproxy -c -f /etc/haproxy/haproxy.cfg
Configuration file is valid
[root@ka1 keepalived]#systemctl restart haproxy.service
ka1主机可以起来
ka2主机起不来，原因是ka2配置文件配的10.0.0.100地址，但实际主机没有10.0.0.100这个地址，服务是起不来的
如何在没有这个地址的情况下把服务起来，需要修改内核参数
查看需要修改的参数
[root@ka2 conf.d]#sysctl -a |grep bind
net.ipv4.ip_nonlocal_bind = 0
修改参数为1，添加在最下面，ka1和ka2节点都需要改
[root@ka2 conf.d]#vim /etc/sysctl.conf
net.ipv4.ip_nonlocal_bind = 1
生效
[root@ka2 conf.d]#sysctl -p
net.ipv4.ip_nonlocal_bind = 1
这样就可以启动服务了
[root@ka2 conf.d]#systemctl restart keepalived.service
第三步：前端客户端解析域名进行访问
[root@rocky8 ~]#curl www.meng.org
web1.meng.org 10.0.0.8
[root@rocky8 ~]#curl www.meng.org
web2.meng.org 10.0.0.18
由于上面创建的down文件，会发现vip漂移到ka2节点上，此时可以把down文件删掉，vip便漂移到ka1上。此时如果挂掉主节点ka1，后面的服务访问是不受影响的
如果挂掉haproxy，会出现半死不活状态
如：停掉haproxy服务
systemctl stop haproxy.service
[root@rocky8 ~]#while : ; do curl www.meng.org;sleep 1;done
curl: (7) Failed to connect to www.meng.org port 80: Connection refused
这种状态需要用脚本来解决（判断服务是否健康，不健康减30）
第一步：写脚本
[root@ka1 keepalived]#cat check_haproxy.sh
#!/bin/bash
killall -0 haproxy
[root@ka1 keepalived]#chmod +x check_haproxy.sh
第二步：调用脚本
[root@ka1 keepalived]#cat conf.d/www.meng.org.conf
vrrp_script check_haproxy {
script "/etc/keepalived/check_haproxy.sh" #脚本执行成功就算了，执行失败说明检查haproxy连不上
interval 1
weight -30 #连不上减30
fall 3
rise 2
timeout 2
}
vrrp_script check_down {
script "[ ! -f /etc/keepalived/down ]"
interval 1
weight -30
fall 3
rise 2
timeout 2
}
vrrp_instance VI_1 {
state MASTER
interface eth1
virtual_router_id 66
priority 100
advert_int 1
#nopreempt
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.150/24 dev eth0 label eth0:1
}
unicast_src_ip 192.168.10.100
unicast_peer{
192.168.10.101
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
track_script {
check_haproxy
}
}
[root@ka1 keepalived]#systemctl restart keepalived.service
ka2同上操作
关闭ka1节点上的haproxy查看优先级是否减下来(减为70后，ka2优先级高于ka1，VIP飘到ka2)
[root@ka1 keepalived]#tcpdump -i eth1 -nn host 192.168.10.100
16:55:00.365235 IP 192.168.10.101 > 192.168.10.100: VRRPv2, Advertisement, vrid 88, prio 100, authtype simple, intvl 1s, length 20
16:55:01.058486 IP 192.168.10.100 > 192.168.10.101: VRRPv2, Advertisement, vrid 66, prio 70, authtype simple, intvl 1s, length 20
16:55:01.366680 IP 192.168.10.101 > 192.168.10.100: VRRPv2, Advertisement, vrid 88, prio 100, authtype simple, intvl 1s, length 20
16:55:02.058994 IP 192.168.10.100 > 192.168.10.101: VRRPv2, Advertisement, vrid 66, prio 70, authtype simple, intvl 1s, length 20
16:55:02.367722 IP 192.168.10.101 > 192.168.10.100: VRRPv2, Advertisement, vrid 88, prio 100, authtype simple, intvl 1s, length 20
16:55:03.060004 IP 192.168.10.100 > 192.168.10.101: VRRPv2, Advertisement, vrid 66, prio 70, authtype simple, intvl 1s, length 20
16:55:03.369209 IP 192.168.10.101 > 192.168.10.100: VRRPv2, Advertisement, vrid 88, prio 100, authtype simple, intvl 1s, length 20
16:55:03.746656 IP 192.168.10.101 > 192.168.10.100: VRRPv2, Advertisement, vrid 66, prio 80, authtype simple, intvl 1s, length 20
[root@rocky8 ~]#while : ; do curl www.meng.org;sleep 1;done
web1.meng.org 10.0.0.101
web2.meng.org 10.0.0.102
curl: (7) Failed to connect to www.meng.org port 80: Connection refused
curl: (7) Failed to connect to www.meng.org port 80: Connection refused
Sorry Server 10.0.0.102
Sorry Server 10.0.0.102实现Nginx 反向代理的高可用
关闭上面饰演的haproxy服务
第一步：ka1、ka2安装nginx
[root@ka1 conf.d]#apt -y install nginx
第二步：ka1、ka2配置nginx反向代理 （ubuntu安装配置文件带有include /etc/nginx/conf.d/*.conf;）
[root@ka1 conf.d]#vim /etc/nginx/conf.d/www.meng.com.conf
upstream www.meng.com {
server 10.0.0.7:80;
server 10.0.0.17:80;
}
server {
listen 10.0.0.200:80;
server_name www.meng.com;
location / {
proxy_pass http://www.meng.com
}
}
[root@ka1 conf.d]#nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@ka1 conf.d]#nginx -s reload访问测试（并没有实现nginx的高可用）
停止一台nginx的服务查看现象（停止ka2）
[root@rocky8 ~]#while : ; do curl www.meng.com;sleep 1;done
web1.meng.com 10.0.0.7
web2.meng.com 10.0.0.17
web1.meng.com 10.0.0.7
web2.meng.com 10.0.0.17
curl: (7) Failed to connect to www.meng.com port 80: Connection refused
curl: (7) Failed to connect to www.meng.com port 80: Connection refused增加脚本，检查nginx进程的健康性（所有keepalived主机都增加并更改）
第一步：
[root@ka2 keepalived]#cat check_nginx.sh （也可以根据状态页存在不存在检查状态）
#!/bin/bash
killall -0 nginx
[root@ka2 keepalived]#chmod +x check_nginx.sh
第二步：
[root@ka2 conf.d]#cat www.meng.com.conf
vrrp_script check_nginx {
script "/etc/keepalived/check_nginx.sh"
interval 1
weight -30
fall 3
rise 2
timeout 2
}
vrrp_script check_down {
script "[ ! -f /etc/keepalived/down ]"
interval 1
weight -30
fall 3
rise 2
timeout 2
}
vrrp_instance VI_2 {
state MASTER
interface eth1
virtual_router_id 88
priority 100
advert_int 1
#nopreempt
authentication {
auth_type PASS
auth_pass 654321
}
virtual_ipaddress {
10.0.0.200/24 dev eth0 label eth0:1
}
unicast_src_ip 192.168.10.101
unicast_peer{
192.168.10.100
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
track_script {
check_nginx
}
}由于ka2的nginx还未开启，再次查看访问结果
curl: (7) Failed to connect to www.meng.com port 80: Connection refused
curl: (7) Failed to connect to www.meng.com port 80: Connection refused
curl: (7) Failed to connect to www.meng.com port 80: Connection refused
web1.meng.com 10.0.0.7
web2.meng.com 10.0.0.17
会发现已正常还可以利用通知脚本,实现切换时,自动重启服务（所有keepalived机器）
[root@ka1 keepalived]#cat notify.sh
#!/bin/bash
contact='[email protected]'
email_send='[email protected]'
email_passwd='bdeuidngbzvufjjf'
email_smtp_server='smtp.qq.com'
. /etc/os-release
msg_error() {
echo -e "\033[1;31m$1\033[0m"
}
msg_info() {
echo -e "\033[1;32m$1\033[0m"
}
msg_warn() {
echo -e "\033[1;33m$1\033[0m"
}
color () {
RES_COL=60
MOVE_TO_COL="echo -en \\033[${RES_COL}G"
SETCOLOR_SUCCESS="echo -en \\033[1;32m"
SETCOLOR_FAILURE="echo -en \\033[1;31m"
SETCOLOR_WARNING="echo -en \\033[1;33m"
SETCOLOR_NORMAL="echo -en \E[0m"
echo -n "$1" && $MOVE_TO_COL
echo -n "["
if [ $2 = "success" -o $2 = "0" ] ;then
${SETCOLOR_SUCCESS}
echo -n $" OK "
elif [ $2 = "failure" -o $2 = "1" ] ;then
${SETCOLOR_FAILURE}
echo -n $"FAILED"
else
${SETCOLOR_WARNING}
echo -n $"WARNING"
fi
${SETCOLOR_NORMAL}
echo -n "]"
echo
}
install_sendemail () {
if [[ $ID =~ rhel|centos|rocky ]];then
rpm -q sendemail &> /dev/null || yum install -y sendemail
elif [ $ID = 'ubuntu' ];then
dpkg -l |grep -q sendemail || { apt update; apt install -y libio-socket-ssl-perl libnet-ssleay-perl sendemail ; }
else
color "不支持此操作系统，退出!" 1
exit
fi
}
send_email () {
local email_receive="$1"
local email_subject="$2"
local email_message="$3"
sendemail -f $email_send -t $email_receive -u $email_subject -m $email_message -s $email_smtp_server -o message-charset=utf-8 -o tls=yes -xu $email_send -xp $email_passwd
[ $? -eq 0 ] && color "邮件发送成功!" 0 || color "邮件发送失败!" 1
}
notify() {
if [[ $1 =~ ^(master|backup|fault)$ ]];then
mailsubject="$(hostname) to be $1, vip floating"
mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
send_email "$contact" "$mailsubject" "$mailbody"
else
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
fi
}
install_sendemail
notify $1
[ $1 = backup ] && systemctl restart nginx
[root@ka1 keepalived]#systemctl restart nginx keepalived.service
停止nginx，会发现VIP会飘过去再飘回来