二进制部署

Prometheus

(1)上传软件包

下载地址:

https://prometheus.io/download/

(2)解压软件包

mkdir /softwares -pv
tar xf prometheus-2.37.6.linux-amd64.tar.gz -C /softwares/

(3)启动prometheus Server

cd /softwares/prometheus-2.37.6.linux-amd64 
./prometheus 

启动脚本

cat > /etc/sysconfig/prometheus <<'EOF'
PROMETHEUS_HOME=/softwares/prometheus-2.37.6.linux-amd64
EOF


cat > /usr/lib/systemd/system/prometheus.service <<'EOF'
[Unit]
Description= prometheus server daemon
After=network.target

[Service]
EnvironmentFile=/etc/sysconfig/prometheus
ExecStart=/softwares/prometheus-2.37.6.linux-amd64/prometheus \
          --config.file=${PROMETHEUS_HOME}/prometheus.yml \
          --web.listen-address=0.0.0.0:9090 \
          --storage.tsdb.path=${PROMETHEUS_HOME}/data \
          --web.max-connections=10 \
          --storage.tsdb.retention.time=15d \
          --log.level=info \
          --web.read-timeout=5m
    
[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl start prometheus
systemctl status prometheus

(5)访问proemetheus Web UI

http://192.168.0.118:9090/

node-exporter

(1)上传软件包

(2)创建工作目录

mkdir /softwares -pv

(3)解压软件包

tar xf node_exporter-1.5.0.linux-amd64.tar.gz -C /softwares

启动脚本

cat > /usr/lib/systemd/system/node-exporter.service <<'EOF'
[Unit]
Description= Node exporter daemon
After=network.target

[Service]
ExecStart=/softwares/node_exporter-1.5.0.linux-amd64/node_exporter \
          --web.listen-address=:9100 \
          --log.level=info
          
    
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl restart node-exporter
systemctl status node-exporter

监控node expoeter节点:

1、修改二进制的prometheus server的配置文件监控node-exporter:

(1)修改配置文件

[root@xiaosun ~ ]# vim /softwares/prometheus-2.37.6.linux-amd64/prometheus.yml
scrape_configs:
  ...
  # 添加如下2个配置组即可。
  - job_name: "node_exporter_108"
    static_configs:
      - targets: ["192.168.0.118:9100"]

 

(2)重启服务

systemctl restart prometheus

2、prometheus 基于文件的方式动态发现节点并监控:

(1)修改配置文件

[root@xiaosun ~ ]#vim /softwares/prometheus-2.37.6.linux-amd64/prometheus.yml 
...
scrape_configs:
  ...
  - job_name: "node_exporter_file_sd"
    file_sd_configs:
    - files:
      - /softwares/prometheus-2.37.6.linux-amd64/config-file/node-exporter.yml
...
[root@xiaosun ~ ]#
[root@xiaosun ~ ]# systemctl restart prometheus.service 

(2)创建配置文件

[root@xiaosun prometheus-2.37.6.linux-amd64]# pwd
/xiaosun/softwares/prometheus-2.37.6.linux-amd64

[root@xiaosun prometheus-2.37.6.linux-amd64]# mkdir config-file
[root@xiaosun prometheus-2.37.6.linux-amd64]# 

[root@xiaosun prometheus-2.37.6.linux-amd64]# vim config-file/node-exporter.yml
[
  {
    "targets": ["10.0.0.109:19100","10.0.0.108:9100"]
  }
]
[root@xiaosun prometheus-2.37.6.linux-amd64]#

使用Grafana展示数据

1、安装grafana:

上传压缩包

yum -y localinstall grafana-enterprise-9.3.6-1.x86_64.rpm 

systemctl enable --now grafana-server.service 

二进制安装pushgateway组件:

上传pushgateway压缩包

(1)创建目录并解压软件包

mkdir /softwares -pv
tar xf pushgateway-1.5.1.linux-amd64.tar.gz -C /softwares/

(2)编写启动脚本

cat > /usr/lib/systemd/system/pushgateway.service <<'EOF'
[Unit]
Description=pushgateway daemon
After=network.target

[Service]
ExecStart=/softwares/pushgateway-1.5.1.linux-amd64/pushgateway \
          --web.listen-address=:9091 \
          --log.level=info
           
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl restart pushgateway
systemctl status pushgateway

二进制安装altermanager组件:

(1)创建工作目录

[root@xiaosun ~]# mkdir -pv /softwares

(2)解压altermanager

[root@xiaosun ~]# tar xf alertmanager-0.25.0.linux-amd64.tar.gz -C /softwares

(3)修改配置文件

[root@xiaosun alertmanager-0.25.0.linux-amd64]# cat alertmanager.yml 
global:
  resolve_timeout: 5m
  smtp_from: '[email protected]'
  smtp_smarthost: 'smtp.qq.com:465'
  smtp_auth_username: '[email protected]'
  smtp_auth_password: 'aicgtgsyfampbhia'
  smtp_require_tls: false
  smtp_hello: 'qq.com'
route:
  group_by: ['alertname']
  group_wait: 5s
  group_interval: 5s
  repeat_interval: 5m
  receiver: 'email'
receivers:
- name: 'email'
  email_configs:
  - to: '[email protected]'
    send_resolved: true
inhibit_rules:
  - source_match:
      severity: 'critical'
    target_match:
      severity: 'warning'
    equal: ['alertname', 'dev', 'instance']
[root@xiaosun alertmanager-0.25.0.linux-amd64]# 

(4)运行altermanager服务

[root@xiaosun alertmanager-0.25.0.linux-amd64]# ./alertmanager 

3、配置prometheus监控altermanager：

(1)修改altermanager的配置文件

[root@xiaosun prometheus-2.37.6.linux-amd64]# vim prometheus.yml 
...
alerting:
  alertmanagers:
    - static_configs:
        - targets:
            - 10.0.0.108:9093

rule_files:
    - "rule/rules.yml"

(2)编写prometheus的规则文件

[root@xiaosun prometheus-2.37.6.linux-amd64]# mkdir rule
[root@xiaosun prometheus-2.37.6.linux-amd64]# 
[root@xiaosun prometheus-2.37.6.linux-amd64]# vim rule/rules.yml
groups:
- name: container-runtime
  rules:
  - alert: container-10.0.0.108
    expr: up{instance="10.0.0.108:9100", job="node_exporter_108"} == 0
    for: 15s
    annotations:
      summary: "{{ $labels.instance }} 已停止运行超过 15s！"

  - alert: container-容器挂啦-2023
    expr: up{instance="10.0.0.109:9100", job="node_exporter_109"} == 0
    for: 5s
    annotations:
      summary: "Duang~ {{ $labels.instance }} 已停止运行超过 15s！"

(3)重启prometheus server服务

[root@xiaosun prometheus-2.37.6.linux-amd64]# systemctl restart prometheus