prometheus钉钉告警

钉钉添加机器人

将webhook复制到脚本中

点一下加签复制保存下后面有用

配置告警脚本

[root@harbor apps]# cat dingding-keywords.sh 
#!/bin/bash
source /etc/profile
#PHONE=$1
#SUBJECT=$2
MESSAGE=$1

/usr/bin/curl -X "POST" 'https://oapi.dingtalk.com/robot/send?access_token=3833c58d9c22741bfd46b822b73418105954b07127629dbd1ae0a5044a125095' \
-H 'Content-Type: application/json' \
-d '{"msgtype": "text",
    "text": {
         "content": "'${MESSAGE}'"
    }
  }'

用脚本测试发送消息

bash dingbash dingding-keywords.sh "node1_cpu%75,alertname=node内存可用大小"

钉钉就可以看到消息了

配置prometheus钉钉告警

安装dingtalk

wget https://github.com/timonwong/prometheus-webhook-dingtalk/releases/download/v2.1.0/prometheus-webhook-dingtalk-2.1.0.linux-amd64.tar.gz -C /apps
tar xvf prometheus-webhook-dingtalk-2.1.0.linux-amd64.tar.gz
ln -sv /apps/prometheus-webhook-dingtalk-2.1.0.linux-amd64 /apps/prometheus-webhook-dingtalk
cd /apps/prometheus-webhook-dingtalk
cp config.example.yml config.yml

编辑config.yml

[root@harbor prometheus-webhook-dingtalk]# cat config.yml 
## Request timeout
# timeout: 5s

## Uncomment following line in order to write template from scratch (be careful!)
#no_builtin_template: true

## Customizable templates path
templates:
  - contrib/templates/legacy/template.tmpl

## You can also override default template using `default_message`
## The following example to use the 'legacy' template from v0.3.0
#default_message:
#  title: '{{ template "legacy.title" . }}'
#  text: '{{ template "legacy.content" . }}'

## Targets, previously was known as "profiles"
targets:
  alertname:
    url: https://oapi.dingtalk.com/robot/send?access_token=3833c58d9c22741bfd46b822b73418105954b07127629dbd1ae0a5044a125095   #webhook地址
    # secret for signature
    secret: SEC5a4e6a712ef3f5478ccf2b6753c84abc0192867628a36f05c3184727b832ffa1 #加签的地址
    mention:
      mobiles: ['15001xxxxx']

配置自动启动

[root@harbor prometheus-webhook-dingtalk]# cat /etc/systemd/system/prometheus-webhook-dingtalk.service 
[Unit]
Descripton=dingtalk
Documentation=https://github.com/timonwong/prometheus-webhook-dingtalk/
After=network.target

[Service]
Restart=on-failure
WorkingDirectory=/apps/prometheus-webhook-dingtalk
ExecStart=/apps/prometheus-webhook-dingtalk/prometheus-webhook-dingtalk --config.file=/apps/prometheus-webhook-dingtalk/config.yml

[Install]
WantedBy=multi-user.target

systemctl restart prometheus-webhook-dingtalk.service
systemctl enable prometheus-webhook-dingtalk.service

修改alertmanager.yml

  - name: 'dingding.alertname'
    webhook_configs:
    - url: 'http://10.211.55.26:8060/dingtalk/alertname/send' #配置dingtalk的地址和端口
      send_resolved: true

重启alertmanager

systemctl restart alertmanager

我这里默认一直都有报警，钉钉已经收到报警了，如果不清楚告警怎么设置的查看一下我之前的博客