Logstash接收udp/tcp数据 python+ udp/tcp +logstash +elasticsearch

Logstash接收udp/tcp数据

背景：在 Logstash数据源为日志文件操作 基础上进行

一、配置文件

1. D:\usr\local\etc\logstash\pipeline1目录下logstash.conf文件配置

input { 
    stdin { } 
    udp {
        host => "0.0.0.0"
        # 从5000端口获取日志
        port => 5000
        type => "udp"
    }
    tcp {
        mode => "server"
        host => "0.0.0.0"
        # 从5044端口取日志
        port => 5000
        # 需要安装logstash-codec-json_lines插件
        codec => json_lines
        type => "tcp"
    }
}
 
output {
	stdout { codec => rubydebug }
    if [type] == "udp" {
        elasticsearch {
            hosts => ["192.168.1.110:9200"]
            index => "logstash-udp-%{+YYYY.MM.dd}"
        }
    }
    if [type] == "tcp" {
        elasticsearch {
            hosts => ["192.168.1.110:9200"]
            index => "logstash-tcp-%{+YYYY.MM.dd}"
        }
    }
    
}

2. D:\usr\local\etc\logstash\config目录下logstash.yml文件配置

http.host: "0.0.0.0"

二、启动logstash

1. 执行命令，启动logstash容器(注：此处UDP/TCP数据接收端口为5000)

docker run -it -v /d/usr/local/etc/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml -v /d/usr/local/etc/logstash/pipeline1:/usr/share/logstash/pipeline --name logstash1 -p 9601:9600 -p 5000:5000/udp -p 5000:5000 logstash:7.7.0

三、数据发送

1. 在D:\usr\local\bin\start目录下创建send_udp.py文件

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import socket  # 网络通信 TCP，UDP
 
# SOCK_DGRAM表示UDP，SOCKET_STREAM表示TCP
client = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
 
while True:
    msg = input('>>: ').strip()
    if not msg:
        continue
 
    client.sendto(msg.encode('utf-8'), ('192.168.1.110', 5000))
 
    if msg == 'close':
        break
client.close()

2. D:\usr\local\bin\start目录下创建send_tcp.py文件

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import socket  # 网络通信 TCP，UDP
 
# SOCK_DGRAM表示UDP，SOCKET_STREAM表示TCP
host = ('127.0.0.1', 5000)
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(host)
 
while True:
    msg = input('>>: ').strip()
    if not msg:
        continue
 
    client.sendto(msg.encode('utf-8'), host)
 
    if msg == 'close':
        break
client.close()

3. 执行udp数据发送脚本，发送数据

python3 send_udp.py

4. 执行tcp数据发送脚本，发送数据

python3 send_tcp.py

四、查看结果

1. udp数据接收结果

2. tcp数据接收结果

从以上结果可以看出配置成功