首页 > 其他分享 >docker-compose安装EFK

docker-compose安装EFK

时间：2023-07-19 17:46:42浏览次数：37  
标签：7.17 filebeat compose EFK kibana efk elasticsearch docker

一、环境

IP 系统 配置 版本
192.168.10.100 Centos7.9 2核4G Docker Compose version v2.19.1、EFK-7.17.11

 


EFK版本是试用版本

二、安装docker环境

yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo
yum makecache fast
yum -y install docker-ce


cat > /etc/docker/daemon.json <<EOF
{
"registry-mirrors":["https://pft7f97f.mirror.aliyuncs.com","https://registry.docker-cn.com","https://docker.mirrors.ustc.edu.cn","https://dockerhub.azk8s.cn","http://hub-mirror.c.163.com"]
}
EOF

systemctl daemon-reload
systemctl start docker

[root@efk efk]# docker compose version
Docker Compose version v2.19.1

三、下载EFK相关镜像

docker pull docker.elastic.co/elasticsearch/elasticsearch:7.17.11
docker pull docker.elastic.co/kibana/kibana:7.17.11
docker pull docker.elastic.co/beats/filebeat:7.17.11

[root@efk efk]# docker images
REPOSITORY                                      TAG       IMAGE ID       CREATED       SIZE
docker.elastic.co/beats/filebeat                7.17.11   b4bef40e4a4a   3 weeks ago   268MB
docker.elastic.co/elasticsearch/elasticsearch   7.17.11   0f404e39b5e6   3 weeks ago   630MB
docker.elastic.co/kibana/kibana                 7.17.11   ff2a71cd3986   3 weeks ago   798MB

四、编辑filebeat.yaml文件

[root@efk efk]# cat filebeat.yaml 
filebeat.inputs:
- type: log
  paths: 
    - '/usr/share/filebeat/logs/*'

processors:
  - decode_json_fields:
      fields: ["message"]
      target: ""
      overwrite_keys: true

output.elasticsearch:
  hosts: ["http://192.168.10.100:9200"]
  indices:
    - index: "filebeat-%{+yyyy.MM.dd}"

setup.kibana:
  host: "http://192.168.10.100:5601"

logging.json: true
logging.metrics.enabled: false

五、部署EFK系统

5.1 创建数据目录

mkdir /data/efk/es/data/nodes -p
chmod -R 777 /data/efk

5.2 编辑docker-compose.yaml文件

[root@efk efk]# cat docker-compose.yml 
version: '3.3'

services:

  elasticsearch:
    image: "docker.elastic.co/elasticsearch/elasticsearch:7.17.11"
    container_name: elasticsearch
    restart: always
    environment:
    - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    - "discovery.type=single-node"
    - "cluster.name=myes"
    - "node.name=jeven"
    # - xpack.security.enabled: "false"
    ulimits:
      memlock:
        soft: -1
        hard: -1 
    networks:
      myefk:
        ipv4_address: 172.29.120.10
        aliases:
        - es
        - jeven
    ports:
    - "9200:9200"
    - "9300:9300"
    volumes:
    - /data/efk/es/data/:/usr/share/elasticsearch/data

  kibana:
    image: "docker.elastic.co/kibana/kibana:7.17.11"
    restart: always
    environment:
      # 注意这里的配置，否则会导致kibana页面不能打开
      ELASTICSEARCH.URL: http://192.168.10.100:9200
      ELASTICSEARCH.HOSTS: '["http:/192.168.10.100:9200"]'
      I18N_LOCALE: zh-CN
    networks:
      myefk:
        ipv4_address: 172.29.120.20
        aliases:
          - kibana
          - kib
    ports:
    - "5601:5601"
    links:
    - "elasticsearch"

  filebeat:
    image: "docker.elastic.co/beats/filebeat:7.17.11"
    restart: always
    networks:
      myefk:
        ipv4_address: 172.29.120.30
        aliases:
          - filebeat
          - fb
    user: root
    command: ["--strict.perms=false"]
    volumes:
    - /data/efk/filebeat.yaml:/usr/share/filebeat/filebeat.yml
    - /var/lib/docker:/var/lib/docker:ro
    - /var/run/docker.sock:/var/run/docker.sock
    links:
    - "elasticsearch"
    - "kibana"

networks:
  myefk:
    driver: bridge
    ipam:
      config:
        - subnet: 172.29.120.0/24

5.3 运行EFK

[root@efk efk]# docker compose up -d
[+] Running 4/4
 ✔ Network efk_myefk         Created                                                                                                                                                                                                   0.3s 
 ✔ Container elasticsearch   Started                                                                                                                                                                                                   0.4s 
 ✔ Container efk-kibana-1    Started                                                                                                                                                                                                   0.8s 
 ✔ Container efk-filebeat-1  Started

# 关闭命令为 docker compose down

[root@efk efk]# docker compose ps
NAME                IMAGE                                                   COMMAND                  SERVICE             CREATED             STATUS              PORTS
efk-filebeat-1      docker.elastic.co/beats/filebeat:7.17.11                "/usr/bin/tini -- /u…"   filebeat            27 minutes ago      Up 27 minutes       
efk-kibana-1        docker.elastic.co/kibana/kibana:7.17.11                 "/bin/tini -- /usr/l…"   kibana              27 minutes ago      Up 27 minutes       0.0.0.0:5601->5601/tcp, :::5601->5601/tcp
elasticsearch       docker.elastic.co/elasticsearch/elasticsearch:7.17.11   "/bin/tini -- /usr/l…"   elasticsearch       27 minutes ago      Up 27 minutes       0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 0.0.0.0:9300->9300/tcp, :::9300->9300/tcp

5.4 查看efk容器日志

[root@efk efk]# docker compose logs |head
elasticsearch  | {"type": "server", "timestamp": "2023-07-19T08:49:09,038Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "myes", "node.name": "jeven", "message": "loaded module [aggs-matrix-stats]" }
elasticsearch  | {"type": "server", "timestamp": "2023-07-19T08:49:09,038Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "myes", "node.name": "jeven", "message": "loaded module [analysis-common]" }
elasticsearch  | {"type": "server", "timestamp": "2023-07-19T08:49:09,038Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "myes", "node.name": "jeven", "message": "loaded module [constant-keyword]" }
elasticsearch  | {"type": "server", "timestamp": "2023-07-19T08:49:09,054Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "myes", "node.name": "jeven", "message": "loaded module [frozen-indices]" }

5.5 测试访问

[root@efk efk]# curl 192.168.10.100:9200
{
  "name" : "jeven",
  "cluster_name" : "myes",
  "cluster_uuid" : "-y4gQ2IvQ_CohEPfppPnSw",
  "version" : {
    "number" : "7.17.11",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "eeedb98c60326ea3d46caef960fb4c77958fb885",
    "build_date" : "2023-06-23T05:33:12.261262042Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

六、访问Kibana服务

6.1 页面访问kibana进入首页

    http://192.168.10.100:5601

 6.2 查看日志信息

1.进入索引管理界面
选左侧打开目录：Managerment---stack managrment ---数据---索引管理

 2.查看filebeta索引信息

 3.创建索引

选择：索引模式---创建索引---设置索引名称--索引时间戳字段--创建索引

 4.搜索日志信息

在主页，选择discover模块位置，根据字段可搜索日志信息

 5.查看日志文件信息

Observability——日志，点击进入

本次搭建引用文章：https://cloud.tencent.com/developer/article/2210662

出现2个问题：

　　1./data/efk/es目录的权限问题，我都改成了777

       2.kibana页面不能访问问题，修改了docker-compose.yaml文件中，下面2个字段：

      ELASTICSEARCH.URL: http://192.168.10.100:9200
      ELASTICSEARCH.HOSTS: '["http:/192.168.10.100:9200"]'

 

标签：7.17,filebeat,compose,EFK,kibana,efk,elasticsearch,docker
From： https://www.cnblogs.com/yangmeichong/p/17566298.html

相关文章

  • docker “no space left on device”
     1、先使用dockerinfo查看docker的信息dockerinfo可以看到docker的根路径是 /var/lib/docker  2、查看docker根路径的磁盘占用率df-Th/var/lib/docker3、清理docker 方法一：删除所有未运行的容器（已经运行的删除不了，未运行的就一起被删除了）dockerrm$(docker......
  • Docker安装的fastdfs基于不同服务器的数据迁移
    首先，基于docker搭建新的fastdfs中间件，参考地址为：https://blog.csdn.net/ming19951224/article/details/126933299然后将原服务器的storage文件夹下的data文件夹进行备份，打包成bak.zip 将bak.zip下载后上传到新服务器的storage文件夹下 使用unzip解压缩bak.zip，然后进入data.......
  • 用docker搭建自己的node服务器集群
    1.配置java环境，并提交镜像：#1.解析jdk的包到/usr/local/java下：#2.并配置java环境[root@07e065a05df9/]#vim/etc/profile.d/my_env.sh#JAVA_HOMEexportJAVA_HOME=/usr/local/java/jdk1.8.0_301/exportPATH=$PATH:$JAVA_HOME/bin[root@07e065a05df9/]#source/e......
  • go通过docker sdk进入容器执行命令
    代码部分packagemainimport( "context" "fmt" "io" "os" "github.com/docker/docker/api/types" "github.com/docker/docker/client")funcmain(){ //设置远程Docker守护进程的地址 remoteDockerURL:=&quo......
  • docker部署mysql服务
    1：拉取镜像dockerpullmysql2：验证拉镜像成功dockerimages3：创建目录mkdir-p/mysql/data/mysql/logs/mysql/confcd/mysql/conftouchmy.cnf4：创建运行容器dockerrun-p3306:3306--namemysql-v/mysql/conf:/etc/mysql/conf.d-v/mysql/logs:/logs-v/mysql......
  • docker安装mariadb
    第一次使用云服务器，发现存在权限问题。添加一个docker用户组将用户加入docker组刷新docker组拉取镜像查看镜像启动容器查看容器......
  • go语言操作docker删除容器
    代码部分packagemainimport( "context" "fmt" "os" "github.com/docker/docker/api/types" "github.com/docker/docker/client")funcmain(){ //设置远程Docker守护进程的地址 remoteDockerURL:="tcp://10.0.0.1......
  • go语言操作docker重启容器
    代码部分packagemainimport( "context" "fmt" "os" "github.com/docker/docker/api/types/container" "github.com/docker/docker/client")funcmain(){ //设置远程Docker守护进程的地址 remoteDockerURL:="tcp:......
  • docker run -itd
    实现“dockerrun-itd”步骤及代码解析1.简介在开始教你如何实现“dockerrun-itd”之前，首先我们需要了解一些相关的背景知识。Docker是一种轻量级的容器化技术，可以快速创建、部署和运行应用程序。通过使用Docker，我们可以将应用程序及其依赖项打包成一个独立的容器，这样就可以......
  • docker run -d --name bitwarden -v /bw-data/:/data/ -p 8800:80 bitwardenrs/se
    DockerRun命令实现Bitwarden容器化引言在现代软件开发和部署中，容器化技术正变得越来越流行。Docker是一个用于构建、发布和运行应用程序的开源平台，它可以将应用程序及其依赖项打包到一个容器中，提供了一种轻便、可移植和可扩展的方式来部署应用程序。在本文中，我们将学习如何使用......

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99