由于最近公司服务频繁出问题,老板很生气,下面的人都很不好过,于是老大让加一下业务监控,来观察线上数据状态。
但是由于qa环境数据量太少,所以自己搭建了一套环境做相关监控,并且写了个脚本模仿生产上的数据,并且做了很多指标,写了一些Grafana的PromQL语法,网上的资料也很少,我就来补充一下,这个下一章再讲。
注意:
docker和docker-compose请自行下载安装
创建相关目录并给予权限,持久化目录需要给777权限,否则容器启动失败
cd ~ && mkdir prometheus && chmod 777 prometheus
cd prometheus && mkdir grafana_data prometheus_data && chmod 777 grafana_data prometheus_data
vim docker-compose.yml
version: "3.7"
services:
node-exporter:
image: prom/node-exporter:latest
container_name: "node-exporter0"
ports:
- "9100:9100"
restart: always
prometheus:
image: prom/prometheus:latest
container_name: "prometheus0"
restart: always
ports:
- "9090:9090"
volumes:
- "./prometheus.yml:/etc/prometheus/prometheus.yml"
- "./prometheus_data:/prometheus"
grafana:
image: grafana/grafana
container_name: "grafana0"
ports:
- "3000:3000"
restart: always
volumes:
- "./grafana_data:/var/lib/grafana"
注:prometheus主机需要部署以上容器,其他被监控主机只需要部署node-exporter
3.prometheus配置文件(根据自己的服务器ip进行修改targets内容)
vim prometheus.yml
global:
scrape_interval: 15s # 默认抓取周期
external_labels:
monitor: 'codelab-monitor'
scrape_configs:
- job_name: 'node-exporter' #服务的名称
scrape_interval: 5s
metrics_path: /metrics #获取指标的url
static_configs:
- targets: ['139.224.224.229:9100'] # 这个为监听指定服务服务的ip和port,需要修改为自己的ip,貌似云服务必须用公网ip
4.docker-compose启动
docker-compose up -d # 挂载目录给权限,不然会启动失败
如果启动报如下的"没有权限"错误:
grafana0 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana0 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana0 | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana0 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana0 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana0 | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana0 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana0 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana0 | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana0 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana0 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
grafana0 | GF_PATHS_DATA='/var/lib/grafana' is not writable.
grafana0 | You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
grafana0 | mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
则需要将挂载目录赋予权限,即:
chmod 777 grafana_data/ prometheus_data/ #个人根据配置文件中具体的挂在目录进行权限修改
5.访问主机的9090端口可以查到Prometheus监控到的数据,访问3000端口是grafana的界面,账密admin/admin
6.配置监控,点击齿轮,选择Data Sources
7.点击Add data source
8.填写prometheus地址 ip+端口或者主机名+端口(需要配置hosts)
9.点击save&test
10.点击+号,选择import
11.输入模板号9276,点击load。 8919
11.导入模板完成