当然,下面是一个完整的示例,包括Filebeat和Logstash的配置,以便实现按项目进行索引拆分的日志收集:
**Filebeat 配置示例:**
```yaml
filebeat.inputs:
- type: log
enabled: true
paths:
- /opt/logs/qfzs-assistant/log_info.log
fields:
project: "assistant"
- type: log
enabled: true
paths:
- /opt/logs/qfzs-shop/log_info.log
fields:
project: "qfzs-shop"
- type: log
enabled: true
paths:
- /opt/logs/qfzs-table/log_info.log
fields:
project: "qfzs-table"
- type: log
enabled: true
paths:
- /opt/logs/qfzs-user/log_info.log
fields:
project: "qfzs-user"
output.logstash:
hosts: ["localhost:5044"]
```
**Logstash 配置示例:**
```conf
input {
beats {
port => 5044
}
}
filter {
if "assistant" in [fields][project] {
mutate {
add_field => { "index_prefix" => "filebeat-assistant" }
}
}
else if "qfzs-shop" in [fields][project] {
mutate {
add_field => { "index_prefix" => "filebeat-qfzs-shop" }
}
}
else if "qfzs-table" in [fields][project] {
mutate {
add_field => { "index_prefix" => "filebeat-qfzs-table" }
}
}
else if "qfzs-user" in [fields][project] {
mutate {
add_field => { "index_prefix" => "filebeat-qfzs-user" }
}
}
else {
mutate {
add_field => { "index_prefix" => "filebeat-default" }
}
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "%{[index_prefix]}-%{+YYYY.MM.dd}"
}
}
```
请确保你在机器上正确安装了Filebeat和Logstash,并将上述配置文件分别保存为`filebeat.yml`和`logstash.conf`。然后按照以下步骤进行配置:
1. 将`filebeat.yml`配置文件放置到Filebeat的配置目录中,通常是`/etc/filebeat/`。
2. 将`logstash.conf`配置文件放置到Logstash的配置目录中,通常是`/etc/logstash/conf.d/`。
3. 确保Logstash已经安装了beats输入插件和elasticsearch输出插件。你可以使用以下命令来安装插件:
```bash
sudo bin/logstash-plugin install logstash-input-beats
sudo bin/logstash-plugin install logstash-output-elasticsearch
```
4. 启动Logstash服务:
```bash
sudo bin/logstash -f /etc/logstash/conf.d/logstash.conf
```
5. 启动Filebeat服务:
```bash
sudo systemctl start filebeat
```
现在,Filebeat将按照配置的路径和项目字段值收集日志,并发送到Logstash的5044端口。Logstash根据项目字段的值将日志事件发送到不同的索引。每个索引的名称格式为`filebeat-projectName-YYYY.MM.dd`,其中`projectName`对应于每个项目的名称。
希望这个完整示例可以帮助你实现按项目进行索引拆分的日志收集。如果有任何进一步的问题,请随时提问。
标签:ELK,filebeat,log,fields,Logstash,拆分,qfzs,日志,logstash From: https://www.cnblogs.com/JIKes/p/17881856.html